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Abstract 


The  purpose  of  this  study  was  to  evaluate  the  success  of  the  tactical  electronic  medical 
record  (Tactical  EMR)  and  its  ability  to  improve  medical  documentation.  The  study  evaluated 
data  collected  from  medical  personnel  using  the  Tactical  EMR.  The  dependent  variable  used  to 
evaluate  success  was  organization  impact  and  the  independent  variables  were  system  quality, 
information  quality,  usage,  user  satisfaction,  individual  impact,  unit,  rank,  duty  position,  and 
time  with  the  unit. 

An  analysis  of  135  survey  responses  revealed  that  users  ranked  information  quality  the 
highest,  followed  by  organizational  impact,  system  quality,  user  satisfaction,  individual  impact, 
and  usage.  The  results  of  a  multiple  linear  regression  were  significant  and  supported  the 
alternate  hypothesis,  which  stated  that  the  level  of  success  will  vary  according  to  the  independent 
variables.  Individual  impact  was  found  as  the  only  significant  predictor  for  organizational 
impact.  A  one-way  ANOVA  test  compared  organizational  impact  scores  by  demographics  and  a 
significant  difference  was  found  among  time  with  the  unit,  particularly  with  those  assigned  to  the 
unit  0  to  12  months. 

The  results  provide  important  feedback  regarding  the  success  of  the  Tactical  EMR.  It 
highlights  critical  areas  for  further  examination  and  development  in  order  to  improve  the  Tactical 
EMR  and  its  future  implementation. 
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Ethical  Considerations 

Throughout  the  course  of  this  study,  no  direct  patient  information  or  personal  information 
was  used.  The  only  identifiers  used  for  this  study  were  unit,  rank,  duty  position,  and  time  with 
the  unit.  These  identifiers  were  used  to  determine  if  there  were  significant  difference  between 
them  and  the  variables  selected  in  the  study.  There  were  no  other  ethical  considerations  for  this 
study. 
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Introduction 

There  has  not  been  a  point  in  history  where  the  demands  on  the  Department  of  Defense 
(DOD)  and  the  scrutiny  over  medical  practice  have  been  more  critical  than  they  are  today.  Three 
important  catalysts  that  have  drawn  considerable  attention  to  each  are  the  terrorist  attacks  of 
September  11,  2001  and  the  results  of  two  Institute  of  Medicine  (IOM)  reports.  Independently 
addressing  the  concerns  surrounding  current  military  operations  or  the  state  of  the  American 
healthcare  system  presents  a  number  of  unique  challenges.  However,  when  addressed 
simultaneously,  the  challenges  increase  tenfold.  The  DOD’s  unique  position  makes  it 
responsible  for  addressing  the  challenges  of  both  military  operations  and  healthcare  concerns 
simultaneously. 

Military  operations  of  yesterday,  those  prior  to  September  11,  2001,  include  operations  like 
Desert  Storm  in  Iraq,  Restore  Hope  in  Somalia,  and  the  ongoing  operations  of  Joint  Endeavor  in 
Bosnia-Herzegovina  and  Joint  Guard  in  Kosovo.  Military  operations  of  today,  post  September 
11,  2001,  include  operations  like  Enduring  Freedom  in  Afghanistan  and  Iraqi  Freedom  in  Iraq. 
Lessons  learned  from  both  the  operations  of  yesterday  and  today  illustrate  that  the  battlefield  is 
no  longer  symmetric  nor  is  the  enemy  clearly  defined.  New  doctrine,  better  technology,  and  a 
sharper  focus  on  protecting  the  military’s  best  asset,  the  Soldier,  Sailor,  Airman,  and  Marine,  are 
required  to  achieve  success  on  the  battlefield  of  tomorrow.  The  key  to  protecting  the  military’s 
best  asset  is  superior  medical  care. 

The  IOM  reports  indicate  that  somewhere  between  44,000  and  98,000  people  die  each  year 
because  of  medical  errors.  Although  the  reports  do  not  clearly  delineate  between  military  and 
civilian  medical  errors,  they  indicate  that  medical  errors  are  a  problem  in  any  setting  and  can 
lower  the  level  of  trust  in  the  healthcare  system  (Institute  of  Medicine  [IOM]  (a),  2000).  The 
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Army  currently  supports  approximately  326,750  Soldiers  deployed  overseas  in  over  120  different 
countries  (Schoomaker  &  Brownlee,  2004).  Maintaining  superior  medical  care  for  each  Soldier, 
especially  those  deployed  in  austere  and  harsh  conditions,  is  a  monumental  task.  It  is  in  these 
conditions  that  the  Soldier’s  reliance  on  the  medical  system  is  at  its  highest.  Trust  between  the 
medic  and  the  Soldier  is  paramount  and  involves  an  uncompromising  relationship  that 
encompasses  medical  care  before,  during,  and  after  military  operations.  To  improve  and 
maintain  trust  in  the  healthcare  system,  the  IOM  endorses  the  use  of  information  technology. 
Information  technology  provides  a  responsive,  integrated,  and  safe  healthcare  system  necessary 
for  providing  quality  healthcare  (Institute  of  Medicine  [IOM]  (b),  2001).  Like  the  IOM,  the 
DOD  also  recognizes  the  importance  of  information  technology.  Information  technology  is  the 
capstone  for  the  Army’s  transformation  and  is  the  solution  for  quality  healthcare  before,  during, 
and  after  military  operations. 

Military  operations  of  yesterday  challenged  the  trust  in  the  Army’s  medical  system.  The 
absence  and  loss  of  medical  documentation  during  military  deployments  has  prevented  the  early 
identification  and  diagnosis  of  deployment  related  illnesses  (Rostker,  1999).  Such  practices  are 
unacceptable  for  today’s  military  operations.  Therefore,  in  addition  to  pre  and  post-deployment 
screening  of  Soldiers,  the  DOD  has  been  developing  a  tactical  electronic  medical  record 
(Tactical  EMR).  A  Tactical  EMR  enables  the  DOD  to  enhance  point-of-care  treatment,  establish 
a  longitudinal  health  record,  maintain  continuity  of  care,  improve  quality  of  care,  improve 
medical  surveillance,  and,  ultimately,  provide  superior  healthcare. 

The  first  Tactical  EMR  to  be  deployed  with  the  Army’s  conventional  forces  is  the  Stryker 
Brigade  Combat  Teams’  (SBCT)  Medical  Information  System,  which  is  currently  deployed  and 
used  in  the  Army’s  Stryker  Brigades.  The  Tactical  EMR  currently  fielded  with  the  SBCTs  is 
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unique  to  the  Western  Regional  Medical  Command  (WRMC)  and  will  serve  as  a  tool  to  guide 
the  future  development  and  implementation  with  other  units  throughout  the  Army.  The  primary 
components  of  the  Western  Regional  Tactical  EMR  include  the  Personal  Information  Carrier 
(PIC),  the  Battlefield  Medical  Information  System-Tactical  (BMIS-T),  the  Composite  Health 
Care  System  II-Theater  (CHCS  ITT),  the  Theater  Medical  Information  Program  (TMIP)  server, 
and  the  Integrated  Clinical  Database  (ICDB)  and  its  server.  Together,  Madigan  Army  Medical 
Center  (MAMC),  the  SBCTs,  TMIP,  Medical  Communications  for  Combat  Casualty  Care 
(MC4),  and  the  Telemedicine  &  Advanced  Technology  Research  Center  (TATRC)  have  been 
working  as  a  team  to  provide  a  seamless  digital  healthcare  infrastructure  for  the  Soldiers  who  are 
supporting  today’s  and  tomorrow’s  military  operations  (R.L.  Barnhill,  personal  communication, 
October  01,  2004). 

Conditions  that  Prompted  the  Study 

In  March  2003,  the  3rd  Brigade,  2nd  Infantry  Division,  was  the  first  SBCT  to  receive  BMIS- 
T.  The  intent  of  the  initial  fielding  of  BMIS-T  was  to  evaluate  its  capabilities.  The  fielding 
consisted  of  the  PICs  and  the  BMIS-T  hand-held  devices.  To  test  the  “proof  of  concept”  and 
evaluate  the  two  components,  electronic  field  medical  cards  were  developed  and  tested  to 
determine  if  they  could  be  passed  through  the  evacuation  chain.  In  April  2003,  the  1st  Brigade, 
25th  Infantry  Division  was  the  second  SBCT  to  receive  BMIS-T.  In  addition  to  BMIS-T, 
MAMC  purchased  laptops  and  desktops  for  the  SBCTs  to  extend  the  “proof  of  concept”  test, 
improve  collaboration  between  MAMC  and  the  SBCTs,  and  to  meet  TMIP  network  certification 
requirements.  These  computers  were  loaded  with  ICDB  web-based  software,  which  enabled  the 
SBCTs  to  transfer  patient  notes  from  their  units  to  MAMC’s  ICDB  database.  MAMC  also 
upgraded  the  BMIS-T  hand-held  device  by  installing  an  injury  prevention  scorecard  and  adding 
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MEDFUSION,  a  program  which  enabled  a  bi-directional  information  exchange  between  BMIS- 
T  and  ICDB  (R.L.  Barnhill,  personal  communication,  September  29,  2004). 

In  August  2003,  the  U.S.  Army  Medical  Department  (AMEDD)  Board  conducted  a  test  of  the 
Tactical  EMR  with  1st  Brigade  and  identified  a  number  of  issues  requiring  further  development. 
Improvements  were  made;  however,  not  in  time  for  3rd  Brigade  to  fully  test  their  system  before 
deploying  to  Iraq.  In  October  2003,  3rd  Brigade  deployed  with  updated,  yet  untested,  equipment 
to  operate  the  Tactical  EMR.  From  October  2003  to  October  2004,  1  st  Brigade  used  the  Tactical 
EMR  in  garrison  and  during  operations  at  the  Joint  Readiness  Training  Center.  In  spring  2004, 
the  operating  system  for  CHCS  II-T  was  upgraded  to  Windows  2000  and  the  system  was 
revalidated.  In  October  2004,  3rd  Brigade  redeployed  back  to  Fort  Lewis  and  1st  Brigade 
deployed  to  Iraq.  1st  Brigade  deployed  with  more  training  and  with  the  latest  upgrades  to  the 
Tactical  EMR  (R.L.  Barnhill,  personal  communication,  September  29,  2004). 

The  Army’s  proposed  architecture  for  the  Tactical  EMR  starts  with  data  collected  at  the 


point-of-injury  (Figure  1). 


Figure  1.  Data  Flow  for  the  Tactical  Electronic  Medical  Record 
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The  Soldier’s  PIC  is  inserted  into  the  BMIS-T  hand-held  device  and  patient  care  is  documented 
and  saved  on  both  devices.  Information  is  then  uploaded  from  the  BMIS-T  hand-held  into  a 
CHCS  II-T  laptop  at  the  battalion  aid  station  (BAS)  or  brigade  support  medical  company 
(BSMC).  The  information  will  then  be  forwarded  to  an  interim  TMIP  database  or  the  Joint 
Medical  Workstation  (JMeWS)  and  then  on  to  a  clinical  data  repository  (CDR).  Data  is 
transferred  at  each  of  the  perspective  levels  through  direct  data  interface  between  systems, 
tactical  radio,  secure  internet,  or  removable  media,  such  as  a  disk.  The  intent  of  the  CDR  is  to 
store  patient  data  and  be  accessible  by  medical  facilities  equipped  CHCS  II  and  Joint  Medical 
Workstation  (JMeWS)  access.  It  is  envisioned  that  the  data  stored  on  the  PIC  and  in  the  CDR, 
will  establish  a  longitudinal  health  record  and  make  a  Soldier’s  medical  information  readily 
available  at  any  echelon  of  medical  care  (Roller  &  Calgani,  2005;  R.L.  Barnhill,  personal 
communication,  March  25,  2005). 

Due  to  project  development  timelines  and  other  technical  considerations,  the  Army  is  not 
able  to  deploy  its  fully  proposed  Tactical  EMR  architecture  at  present.  Individual  components 
are  currently  being  tested  and  used,  but  are  not  fully  integrated  into  one  consolidated 
architecture.  The  Tactical  EMR  being  tested  in  the  Western  Region  is  a  variation  to  the  Army’s 
proposed  architecture.  It  capitalizes  on  a  combination  of  elements  from  both  the  Army’s 
architecture  and  systems  currently  in  use  at  MAMC.  The  Western  Regional  Tactical  EMR 
enables  components  of  the  Army’s  architecture  to  be  evaluated  and  it  provides  the  Stryker 
Brigades  with  the  tools  necessary  (i.e.  ICDB)  to  establish  a  Tactical  EMR  for  their  Soldiers. 

The  Western  Regional  Tactical  EMR  operates  similar  to  the  Army’s  proposed  architecture, 
but  it  streamlines  the  flow  of  information  and  employs  the  use  of  ICDB  (Figure  1).  The 
Soldier’s  PIC  is  inserted  into  the  BMIS-T  hand-held  device  and  patient  care  is  documented  and 
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saved  on  both  devices.  Information  is  uploaded  from  the  BMIS-T  hand-held  one  of  two  ways: 
into  the  CHCS  II-T  laptop  or  into  the  ICDB  laptop.  If  the  information  is  uploaded  into  the 
CHCS  II-T  laptop  at  the  BAS  or  BSMC,  the  information  is  forwarded  to  Madigan’s  TMIP 
server,  which  then  forwards  the  information  directly  to  the  Madigan  ICDB  server.  Data  is 
transferred  at  each  of  the  perspective  levels  through  direct  data  interface  between  systems, 
tactical  radio,  secure  internet,  or  removable  media,  such  as  a  disk.  If  the  information  is  uploaded 
into  the  ICDB  laptop,  it  can  be  fed  directly  into  the  Madigan  server  through  any  secure  internet 
connection.  Unlike  the  CHCS  II-T  data  flow,  the  ICDB  connection  speeds  the  flow  of 
information  and  allows  reach-back  visibility  of  all  medical  data  stored  in  ICDB  from  the  combat 
theater  or  garrison.  As  patient  encounters  populate  ICDB,  both  garrison  and  field  providers  with 
internet  connectivity  have  access  to  the  most  up-to-date  medical  history  for  Soldiers  receiving 
medical  treatment  at  the  various  echelons  of  medical  care  (R.L.  Barnhill,  personal 
communication,  September  29,  2004).  To  date,  there  has  been  little  literature  to  evaluate  the 
success  of  the  Tactical  EMR. 

Statement  of  the  Problem 

The  Tactical  EMR  has  been  operational  for  over  a  year.  During  this  time,  the  two  SBCTs 
have  had  the  opportunity  to  thoroughly  evaluate  the  system  and  identify  its  strengths  and 
weaknesses.  To  effectively  analyze  the  success  of  the  Tactical  EMR’s  implementation  and  its 
ability  to  improve  medical  documentation,  it  is  important  to  evaluate  it  from  a  number  of 
perspectives.  One  important  perspective  is  to  evaluate  success  from  the  user  or  stakeholder 
perspective.  Feedback  from  the  users  allows  leadership  to  assess  the  implementation,  dataflow 
processes,  and  helps  determine  if  it  is  meeting  user  needs.  Timely  feedback  and  a  thorough 
analysis  of  user  satisfaction  will  help  identify  strengths,  weaknesses,  barriers,  and  deficiencies. 
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Early  identification  of  issues  allows  leadership  to  correct  problems  in  a  timely  manner,  provides 
valuable  insight  for  implementation  with  other  units,  and  ensures  Soldiers  continue  to  receive 
superior  medical  care. 

In  August  2003,  an  assessment  of  the  Tactical  EMR  was  conducted  by  AMEDD  Board.  The 
assessment  evaluated  the  effectiveness,  suitability,  and  survivability  of  the  system  (U.S.  Army 
Medical  Department  Board,  2003).  As  part  of  the  assessment,  the  Board  conducted  a  limited 
survey  to  get  user  feedback.  The  survey  sampled  eight  people  and  included  only  those 
participating  in  the  assessment.  This  sample  was  small  and  did  not  accurately  represent  the 
current  users  who  are  operating  in  a  variety  of  healthcare  environments. 

Literature  Review 

Gulf  War  illness  and  its  cause(s)  has  been  the  subject  of  much  debate  over  the  past  decade. 
The  crux  of  the  debate  lies  with  lack  of  medical  documentation,  lost  medical  records,  and  the 
inability  to  determine  the  root  cause(s)  of  the  illness.  Roughly  697,000  military  personnel 
deployed  in  support  of  military  operations  in  the  Persian  Gulf  and  little  to  no  documentation 
exists  to  identify:  (1)  the  names  and  locations  of  deployed  personnel;  (2)  exposure  to 
environmental  health  hazards;  (3)  changes  in  health  status  of  deployed  personnel;  and  (4)  records 
of  immunizations  and  other  health  services  provided  while  deployed  (Government  Accounting 
Office  [GAO]  (a),  1997,  p.  3).  The  result  is  a  large  number  of  Soldiers  with  unexplained  medical 
conditions  that  are  unable  to  be  properly  diagnosed  or  treated.  The  Presidential  Advisory 
Committee  on  Gulf  War  Veterans’  Illnesses  ultimately  concluded  that  “many  of  the  health 
concerns  of  Gulf  War  veterans  may  never  be  fully  resolved  because  of  lack  of  data”  (GAO  (a), 
1997,  p.  1). 
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To  identify  the  medical  recordkeeping  processes  during  and  after  the  first  Gulf  War,  the 
Office  of  the  Special  Assistant  to  the  Deputy  Secretary  of  Defense  for  Gulf  War  Illnesses 
conducted  a  study.  In  their  study  they  identified  a  number  of  issues  that  affected  medical 
documentation  and  medical  recordkeeping  processes.  One  important  issue  identified  was 
military  policy,  which  played  an  important  role  in  complicating  the  medical  recordkeeping 
practices.  Policies  that  existed  before  the  first  Gulf  War  were  written  to  support  peacetime 
health  services  and  did  not  address  deployments,  especially  one  as  rapid  and  large-scale  as  the 
first  Gulf  War.  Units  that  deployed  found  themselves  unprepared  to  deal  with  the  requirements 
of  maintaining  health  records  in  a  battlefield  environment.  Throughout  the  DOD,  policies  were 
service  specific  and  recordkeeping  practices  ranged  from  units  deploying  with  a  portion  of  the 
health  record,  with  the  entire  health  record,  or  with  no  health  record  at  all.  Unclear  guidance  and 
questions  regarding  operational  security  also  confused  the  issue  of  documentation  for 
investigational  vaccines  used  during  operations.  The  end  results  were  incomplete,  altered,  and/or 
lost  medical  records  (Rostker,  1999). 

In  1997,  the  Government  Accounting  Office  (GAO)  was  tasked  by  Congress  to  review  the 
completeness  of  Gulf  War  medical  records  and  to  conduct  an  analysis  of  medical  surveillance 
procedures  completed  for  Operation  Joint  Endeavor  in  Bosnia-Herzegovina,  Croatia,  and 
Hungary,  1995-1996.  The  goal  was  to  determine  if  corrections  had  been  made  since  the  first 
Gulf  War  and  to  see  if  health  related  safeguards  were  in  place  to  protect  Soldiers.  Medical 
surveillance  procedures  examined  in  the  study  were  deployment  information,  environmental 
health  assessment  and  disease  monitoring,  medical  assessments,  and  medical  recordkeeping. 
Findings  revealed  that  24%  of  personnel  did  not  receive  in-theater  post-deployment  medical 
assessments,  21%  did  not  receive  home  station  post-deployment  medical  assessments,  and  32% 
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did  not  receive  a  tuberculin  skin  test.  Both  the  centralized  database  and  the  medical 
recordkeeping  processes  were  incomplete.  Medical  documentation  for  in-theater  medical 
assessments,  vaccines,  and  battalion  aid  station  visits  were  found  to  be  missing  in  medical 
records.  Two  factors  that  contributed  to  the  results  were  that  service  members  were  allowed  to 
hand  carry  in-theater  medical  assessments  back  to  home  station  and  that  paper-based  records 
systems  were  prone  to  lost  or  misplaced  medical  documentation.  A  recommendation  cited  in  the 
report  to  address  medical  documentation  issues  was  the  development  of  computerized  medical 
records  (GAO  (a),  1997). 

To  improve  medical  surveillance  and  medical  recordkeeping  practices,  Congress  added  a 
provision  in  the  Defense  Authorization  Act  for  1998.  The  provision  mandated  the  establishment 
of  a  medical  tracking  system  for  service  members  deployed  overseas  as  part  of  contingency  or 
combat  operations.  Elements  of  the  system  would  include  “use  of  pre-deployment  medical 
examinations  and  post-deployment  medical  examinations  to  accurately  record  the  medical 
condition  of  members  before  their  deployment  and  any  changes  in  their  medical  condition  during 
the  course  of  their  deployment”  (Government  Accounting  Office  [GAO]  (b),  2003,  p.6). 
Recordkeeping  procedures  would  include  “results  of  all  medical  examinations  conducted  under 
the  system,  records  of  all  healthcare  services  (including  immunizations)  received  by  members. . . 
and  that  records  of  events  occurring  in  the  deployment  area  that  may  affect  the  health  of  such 
members  shall  be  retained  and  maintained  in  a  centralized  location  to  improve  future  access  to 
records”  (GAO  (b),  2003,  p.  7). 

In  September  2003,  the  GAO  released  a  follow  up  report  that  again  evaluated  DOD  changes 
and  compliance  with  Force  Health  Protection  (FHP)  initiatives.  FHP  initiatives  were  designed 
and  implemented  to  address  all  health  related  activities  developed  to  protect  the  “Total  Force” 
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(Deployment  Health  Support  Directorate,  2003).  The  GAO  report  examined  the  health  related 
activities  for  deployments  supporting  Operation  Enduring  Freedom  and  Operation  Joint  Guard 
and  found  that  among  the  military  installations  visited,  38%  to  98%  of  one  or  both  pre-  and  post¬ 
deployment  forms  were  missing  for  Army  and  Air  Force  personnel.  Missing  immunization 
documentation  ranged  anywhere  from  14%  to  46%,  and  the  centralized  deployment  record 
database  was  found  to  be  missing  vital  medical  information.  A  review  of  battalion  aid  station 
sign-in  logs  revealed  that  39%  of  visits  to  one  aid  station  and  94%  to  another  were  not 
documented  in  individual  health  records.  The  primary  reason  cited  for  the  missing 
documentation  was  that  paperwork  had  been  lost  somewhere  between  the  theater  of  operations 
and  the  home  station.  Although  the  DOD  had  made  important  improvements  since  the  Gulf 
War,  the  GAO  results  indicated  that  the  DOD  was  not  completely  in  compliance  with  FHP  and 
medical  surveillance  policies  (Government  Accounting  Office  (c),  2003).  Noncompliance  with 
these  policies  poses  a  serious  challenge  to  the  early  identification  and  intervention  of  future 
health  problems,  presents  additional  challenges  to  the  medical  mission,  and  stresses  the 
foundation  of  trust  between  the  Soldiers  and  the  military  medical  system. 

The  DOD,  however,  was  not  operating  in  a  vacuum  when  it  worked  to  implement 
improvements  in  medical  surveillance  and  medical  recordkeeping.  Three  significant  events  that 
critically  affected  DOD  operations  were  the  Army  transformation  and  two  reports  published  by 
the  IOM  addressing  patient  safety  and  the  quality  of  U.S.  healthcare.  Both  events  required  the 
DOD  to  make  significant  changes  in  both  the  practice  of  warfare  and  medicine. 

The  Army’s  transformation  was  initiated  by  the  Army  Chief  of  Staff  in  1999.  The  end  of  the 
Cold  War,  an  increased  threat  of  terrorism,  a  growing  number  of  peacekeeping  operations,  and  a 
need  to  respond  to  smaller  conflicts  prompted  a  dramatic  redesign  of  the  way  the  Army 
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conducted  military  operations.  New  concepts,  capabilities,  organizational  structures,  and 
doctrine  were  envisioned  to  meet  the  Nation’s  security  challenges  of  the  21st  century.  The  Army 
transformation  plan  involved  a  comprehensive  transition  from  the  Legacy  Force,  today’s  Army, 
to  the  Objective  Force,  tomorrow’s  Army.  An  Interim  Force  was  introduced  as  a  stop-gap 
measure  to  enable  the  Army  transformation  to  progress  efficiently,  yet  simultaneously  allow  the 
Army  to  respond  to  the  Nation’s  threats.  The  Interim  Force  also  provided  a  well  defined  group 
to  test  and  evaluate  new  concepts,  capabilities,  and  war-fighting  doctrine.  Lessons  learned  from 
the  Interim  Force  would  be  applied  to  development  of  the  Objective  Force.  Fort  Lewis, 
Washington  was  selected  as  the  test  site  to  field  two  Interim  Brigade  Combat  Teams  or  SBCTs 
(Government  Accounting  Office  (d),  2001).  The  first  SBCT,  the  3rd  Brigade,  2nd  Infantry 
Division,  was  projected  to  be  operational  in  2003  and  the  target  date  for  the  second  SBCT,  the 
1st  Brigade,  25th  Infantry  Division,  was  2004  (Vick,  Orletsky,  Pimie,  &  Jones,  2002). 

The  SBCTs  are  so  named  because  the  Stryker  vehicle  is  the  primary  combat  and  combat 
support  platform  used  by  the  units.  The  units  are  designed  to  be  lightweight,  agile,  fast,  and 
prepared  for  rapid  deployment.  The  SBCT  design  allows  it  to  engage  in  a  wide  array  of 
operational  environments,  ranging  from  low  to  high  intensity  conflicts.  Key  SBCT  capabilities 
include:  (1)  increased  operational  and  tactical  mobility;  (2)  enhanced  situational  awareness  and 
understanding;  (3)  combined  arms  integration  down  to  the  company  level;  and  (4)  lethal  and 
non-lethal  joint  effects  (Army  Transformation  (SBCT),  2003).  The  SBCTs  uniqueness  stems 
from  the  large  number  of  organic  units  assigned  to  them.  A  typical  SBCT  includes  three  infantry 
battalions,  a  reconnaissance,  surveillance  and  target  acquisition  cavalry  squadron,  a  brigade 
support  battalion,  a  field  artillery  battalion,  a  military  intelligence  company,  a  signal  company, 
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an  anti-tank  company,  and  a  headquarters  company  (Army  Transformation  (SBCT),  2003;  Vick, 
Orletsky,  Pimie,  &  Jones,  2002). 

Medical  support  for  the  SBCT  is  also  unique.  Combat  health  support  for  the  SBCTs  requires 
important  changes  from  traditional  methods.  In  addition  to  a  physical  redesign  of  SBCT  medical 
units  and  their  support  requirements,  medical  personnel  need  to  be  more  proficient  and  capable 
of  providing  patient  care  in  more  austere  environments.  Patient  care  and  medical  documentation 
becomes  even  more  critical  and  challenging  in  units  that  have  such  a  diverse  missions  like  the 
SBCTs  (D.R.  Davis,  personal  communication,  September  28,  2004). 

Leveraging  technology  is  the  key  to  success  for  the  Army’s  transformation.  This  is  apparent 
in  the  SBCT’s  ability  to  see,  monitor,  and  control  the  battlefield.  The  advanced  technologies 
employed  throughout  the  SBCTs  provide  the  capability  to  leverage  Command,  Control, 
Computer,  Communications,  Intelligence,  Surveillance,  and  Reconnaissance  (C4ISR)  (Army 
Transformation  (SBCT),  2003).  Advanced  technologies,  like  the  SBCT  Medical  Information 
System,  deployed  with  SBCT  medical  units  greatly  enhance  field  medical  capabilities, 
significantly  improve  the  ability  to  conserve  the  fighting  strength,  and  reinforce  trust  in  the 
military  healthcare  system. 

Influencing  DOD  actions  and  validating  the  need  for  improved  information  technology  and 
trust  are  the  IOM’s  two  reports,  which  address  patient  safety  and  quality  healthcare.  The  first 
reports  claims  that  medical  errors  cause  between  44,000  and  98,000  deaths  per  year.  Patients 
who  do  not  die  may  be  left  with  significant  pain,  trauma,  discomfort,  or  dissatisfaction  as  a  result 
of  non-lethal  errors.  Medical  errors  can  occur  in  any  number  of  settings  and  the  costs  associated 
with  them  are  both  financial  and  non-financial.  Perhaps  the  most  significant  non-financial  cost  is 
trust  in  the  healthcare  system  (IOM  (a),  2000).  Trust  in  the  healthcare  system  as  a  whole  is 
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important  and  should  be  addressed  because  as  Soldiers  transition  out  of  the  military  healthcare 
system,  they  will  seek  healthcare  in  civilian  facilities,  and  like  the  DOD,  civilian  facilities  are 
just  as  susceptible  to  medical  record  documentation  problems  and  issues. 

Dr.  Jason  Reider  (2003)  cited  a  study  by  Paul  Tang  in  1994,  which  identified  that  paper 
records  were  missing  in  81%  of  clinical  encounters.  Another  study  stated  that  doctors  were 
unable  to  access  a  patient’s  medical  records  for  30%  of  patient  visits.  The  article  further  stated 
that  27%  of  the  time  the  patient’s  chief  complaint  was  not  recorded  in  the  medical  record  and 
40%  of  the  time  the  patient’s  diagnosis  was  not  recorded.  Physicians  spent  38%  of  their  time 
writing,  dictating,  and  searching  for  information  in  medical  records  (Hultman,  2002).  Bates, 
Ebell,  Gotlieb,  Zapp,  &  Mullins  (2003)  indicate  that  “physicians  have  about  8  unanswered 
questions  for  every  10  ambulatory  visits”  (p.  4).  As  with  the  DOD,  limited  access  to  patient 
information  results  in  duplicative  care,  increased  costs,  decreased  efficiency,  and  distrust  of  the 
health  system.  System  flaws  and  inefficiencies  negatively  impact  all  three  aspects  of  the  “iron 
triangle”:  cost,  quality,  and  access  (IOM  (a),  2000). 

According  to  the  IOM,  the  U.S.  healthcare  system  is  highly  fragmented  and  in  need  of  a 
redesign.  Redesign  of  the  system  includes  the  use  of  information  technology.  “Automation  of 
clinical,  financial,  and  administrative  transactions  is  essential  to  improving  quality,  preventing 
errors,  enhancing  consumer  confidence  in  the  health  system,  and  improving  efficiency”  (IOM 
(b),  2001,  p.  16).  According  to  Tommy  Thompson  and  Dr.  David  Brailer  (2004),  “Health 
information  technology  has  the  potential  to  transform  healthcare  delivery,  bringing  healthcare 
where  it  is  needed  and  refocusing  healthcare  around  the  consumer”  (p.  H). 

Executive  Order  13335,  signed  by  President  Bush  on  April  27,  2004,  highlights  the 
importance  of  adopting  the  electronic  health  record  (EHR).  President  Bush’s  goal  is  for 
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“widespread  adoption  of  interoperable  EHR’s  within  10  years”  (Thomson  &  Brailer,  2004,  p.  A). 
Interoperability  is  crucial  because  competition  within  healthcare  will  produce  a  wide  variety  of 
electronic  medical  record  systems.  To  varying  degrees,  EHRs  are  capable  of  preventing  medical 
errors,  conserving  resources,  reducing  variability  in  care,  increasing  consumer  interaction,  and 
improving  privacy.  Their  full  capability,  however,  cannot  be  realized  until  there  is 
interoperability  between  systems  (Thomson  &  Brailer,  2004). 

The  DOD  is  one  of  the  largest  healthcare  delivery  systems  within  the  U.S.  and  for  many 
years  it  has  used  a  wide  variety  of  health  information  technologies  to  provide  services  to  its 
beneficiaries.  The  closed  nature  of  the  military  health  system  enables  interoperability  among 
many  of  its  medical  systems.  For  example,  the  Composite  Health  Care  System  I  (CHCS  I)  is  an 
automated  information  support  system,  which  enables  physicians  to  access,  store,  and  query 
patient  information.  CHCS  I  automates  many  of  the  healthcare  functions  required  for  daily 
patient  care  and  interfaces  with  over  40  clinical  and  administrative  systems  (Clinical  Information 
Technology  Program  Office,  2004).  System  upgrades  and  improvements  have  led  to  the 
development  DOD’s  newest  system,  CHCS  II.  The  DOD’s  experience  with  health  information 
systems  is  crucial  to  supporting  the  development  of  a  nationwide,  interoperable  EHR  system, 
especially  in  rural  areas  (Thompson  &  Brailer,  2004). 

One  method  to  expand  the  reach  of  electronic  medical  records  to  rural  areas  is  through  the 
use  of  hand-held  medical  devices.  Both  the  DOD  and  civilian  healthcare  facilities  have  recently 
begun  to  employ  hand-held  devices  in  patient  care  activities.  Hand-held  devices  have  a  number 
of  definitions;  perhaps  the  most  common  name  is  the  personal  digital  assistant  (PDA). 
Improvements  in  technology  have  enabled  hand-held  devices  to  support  many  medical 
applications.  This  has  allowed  medical  providers  to  bring  the  most  up-to-date  electronic 
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information  where  it  is  needed  most:  to  the  point-of-care  (Briggs,  2002).  Medical  hand-held 
devices  are  lightweight,  mobile,  capable  of  running  a  variety  of  programs  (each  tailored  to  user 
needs),  and  provide  access  to  electronic  drug  and  medical  reference  material  (Rosenbloom, 
2003).  Although  the  interest  for  hand-held  medical  devices  has  grown,  few  quantitative  studies 
exist  that  evaluate  their  performance  or  impact  on  healthcare. 

Many  qualitative  studies  indicate  that  hand-held  devices  save  time,  improve  billing  precision, 
eliminate  transcription  errors,  increase  medical  documentation,  and  cut  costs.  One  institution 
measured  their  return  on  investment  by  the  reduction  in  the  amount  of  paperwork  processed 
(Briggs,  2002;  Bird,  Zarum  &  Renzi,  2001).  Claudia  Tessier  of  Mobile  Healthcare  Alliance 
(2003)  stated  that,  “mobile  health  care  can  improve  clinical  decision  making,  enhance  quality  of 
care,  reduce  medical  errors,  improve  access  and  exchange  of  information,  improve  accuracy, 
diminish  redundancy,  diminish  paperwork,  increase  time  spent  with  the  patient,  and  improve 
integration”  (p.  2).  Results  from  a  study  on  PDAs  and  point-of-care  trauma  documentation 
revealed  that  the  hand-held  devices  improved  the  efficiency  of  daily  rounds  and  eliminated  the 
need  for  daily  dictation.  PDAs  also  reduced  the  need  to  write  daily  notes,  which  allowed 
physicians  and  nurses  more  time  for  other  patient  care  activities  (Eastes,  2001).  Of  the  literature 
available,  few  studies  addressed  the  capabilities  of  hand-held  medical  devices  and  electronic 
medical  records  in  rural  environments. 

The  DOD’s  worldwide  mission  requires  it  to  operate  in  both  rural  environments  and  areas 
with  poor  medical  infrastructures.  This  provides  the  DOD  a  unique  opportunity  to  evaluate  its 
hand-held  medical  devices  and  electronic  medical  record  capabilities  in  such  areas.  Issues  the 
DOD  must  keep  in  mind  when  developing  and  testing  these  systems  are:  (1)  Force  Health 
Protection  initiatives,  (2)  medical  surveillance  procedures,  (3)  Army  transformation,  and  (4)  the 
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need  to  provide  continuous,  high  quality  healthcare  for  its  Soldiers.  To  address  these  issues,  the 
DOD  combined  the  capabilities  of  an  electronic  medical  record  and  a  hand-held  medical  device 
to  create  its  proposed  architecture  for  a  Tactical  EMR.  To  maximize  the  potential  of  the  Tactical 
EMR  and  test  it  in  real  world  operations,  the  Tactical  EMR  was  fielded  with  the  Army’s  3rd 
Brigade,  2nd  Infantry  Division  and  the  1st  Brigade,  25th  Infantry  Division.  The  Tactical  EMR 
fielded  to  the  Stryker  Brigades  is  unique  to  the  Western  Region  and  consists  of  components  of 
the  Army’s  proposed  architecture  and  systems  currently  in  use  at  MAMC. 

The  primary  components  of  Western  Regional  Tactical  EMR  are  the  PIC,  BMIS-T,  CHCS  II- 
T,  the  TMIP  server,  and  ICDB.  The  PIC  is  an  identification  tag  sized  memory  device  capable  of 
storing  an  individual  Soldier’s  personal  and  medical  information.  It  can  be  carried  next  to  the 
identification  tags  and  is  capable  of  maintaining  a  Soldier’s  complete  medical  history.  All 
medical  encounters,  both  inpatient  and  outpatient,  can  be  recorded  enabling  medical  personnel  to 
have  the  most  up-to-date  information  on  the  Soldier.  As  PIC  capabilities  continue  to  improve,  it 
will  eventually  replace  the  Department  of  Defense  Form  1380  (Field  Medical  Card)  and  the 
Standard  Form  600  (Chronological  Record  of  Medical  Care).  The  PIC  is  capable  of  interfacing 
with  BMIS-T,  CHCS  II-T,  and  ICDB  software  (Barnhill,  2003;  R.L.  Barnhill,  personal 
communication,  September  29,  2004). 

Since  its  inception,  the  PIC  has  undergone  a  series  of  revisions  and  tests.  The  newer  version 
of  the  PIC,  called  the  Electronic  Information  Carrier  (EIC),  eliminates  the  need  for  an  adapter 
and  introduces  wireless  capability.  Medics  will  be  able  to  update  and  store  medical 
documentation  to  the  EIC  without  being  directly  linked.  As  use  of  the  PIC/EIC  increases, 
medical  facilities  throughout  the  different  echelons  of  care  will  have  the  capability  of  viewing 
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and  updating  a  Soldier’s  medical  history.  The  first  EIC  prototypes  are  expected  sometime  in  late 
2005  (Fleming-Michael,  2005). 

BMIS-T  was  originally  developed  in  1995  by  Tommy  Morris  at  the  Telemedicine  and 
Advanced  Technology  Research  Center  (TATRC)  (Fleming-Michael,  2003).  “BMIS-T  is  a 
lightweight,  point-of-care  hand-held  device  that  enables  healthcare  providers  to  record,  store, 
print,  retrieve,  and  transmit  patient  encounters”  (Telemedicine  &  Advanced  Technology 
Research  Center,  2003,  p.  1).  Goals  of  the  BMIS-T  program  include:  (1)  optimizing  patient  care; 
(2)  establishing  a  longitudinal  patient  record;  (3)  facilitating  medical  health  surveillance;  and  (4) 
developing  a  point  of  care  hand-held  device  to  improve  healthcare  by  improving  medical 
decision  making  and  reducing  errors  at  the  first  responder  level.  BMIS-T  extends  the  reach  of 
the  Tactical  EMR  to  the  battlefield  where  the  first  responders  perform  combat  medicine.  In 
addition  to  the  decision  support  tools  within  BMIS-T,  the  system  can  also  maintain  a  large 
library  of  reference  material.  Loading  BMIS-T  with  needed  medical  references  eliminates  the 
need  of  deploying  with  a  large  collection  of  paper  references.  BMIS-T  interfaces  with  the  PIC, 
CHCS  II-T,  the  TMIP,  and  MC4  systems.  MEDFUSION,  a  software  program  developed  by 
MAMC,  enables  BMIS-T  to  interface  with  ICDB  (U.S.  Army  Medical  Department  Board,  2003; 
Telemedicine  &  Advanced  Technology  Research  Center,  2003;  R.L.  Barnhill,  personal 
communication,  September  29,  2004). 

CHCS  II-T  is  the  theater  or  field  extension  of  the  garrison  Composite  Health  Care  System  II. 
CHCS  II-T  operates  in  a  deployed  environment  on  stand-alone  laptop  computers.  It  will  allow 
remote  providers  to  access,  document,  and  store  health  information  at  all  levels  care  within  the 
military  health  system  (Thompson  and  Brailer,  2004).  CHCS  II-T  operates  under  TMIP,  which 
is  “responsible  for  moving  data  through  tactical  communications  to  a  theater  wide 
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database. .  .Data  from  the  theater  database  will  be  fed  into  the  military  health  system’s  Clinical 
Data  Repository  used  for  non-deployed  medical  care”  (Office  of  the  Assistant  Secretary  of 
Defense  (Health  Affairs)  &  TRICARE  Management  Activity,  2004,  p.  3). 

ICDB  was  originally  developed  by  Wilford  Hall  Medical  Center  as  an  outpatient  clinical 
support  system.  The  system  extracts  information  from  CHCS  I  and  provides  clinical  data  and 
caseload  information  for  both  healthcare  providers  and  administrators.  Applications  are  user- 
friendly  and  data  is  presented  through  a  web-base  graphical  user  interface.  Unlike  CHCS  I  and 
CHCS  II,  ICDB  is  accessible  through  a  web  browser  and  is  accessible  anywhere  there  is  an 
internet  connection.  Providers  can  update  patient  information  in  real  time  and  view  the  patient’s 
complete  medical  history.  ICDB  has  a  unidirectional  information  exchange  with  CHCS  I,  the 
Ambulatory  Data  System,  and  the  Clinical  Information  System  and  it  interfaces  with  the  PIC, 
BMIS-T,  and  TMIP  server  (U.S.  Army  Medical  Department  Board,  2003;  R.L.  Barnhill, 
personal  communication,  September  29,  2004). 

In  September  2003,  the  AMEDD  Board  conducted  an  assessment  of  the  SBCT’s  Medical 
Information  System  or  Tactical  EMR.  The  purpose  was  to  evaluate  the  documentation  and  data 
transfer  of  patient  encounters  from  the  point-of-care  in  a  tactical  environment  through  the 
echelons  of  medical  care  to  MAMC.  The  assessment  was  performed  during  a  brigade 
communication  exercise  with  the  1st  Brigade,  25th  Infantry  Division.  The  system’s  capabilities 
were  assessed  in  both  a  tactical  and  garrison  environment.  The  desired  end  state  for  the  Tactical 
EMR  was  for  data  to  flow  from  BMIS-T  to  CHCS  II-T,  through  the  TMIP  server  at  MAMC,  and 
on  to  the  ICDB  server  at  MAMC.  Once  the  patient  encounter  reached  the  ICDB  server,  it  would 
be  visible  for  garrison  providers  at  MAMC.  At  the  completion  of  their  assessment,  the  AMEDD 
board  concluded  that  the  Tactical  EMR  was  not  yet  suitable  and  it  was  ineffective  in  regards  to 
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the  critical  assessment  issues  evaluated.  Items  addressed  under  suitability  were  training 
shortfalls  and  equipment  shortages.  The  system  was  labeled  ineffective  because  “all  medical 
encounters  from  the  company  medic,  brigade  support  medical  company,  troop  medical  clinic, 
and  MAMC  could  not  be  viewed  through  ICDB”  (U.S.  Army  Medical  Department  Board,  2003, 
p.  2-19).  The  assessment  also  included  a  BMIS-T  user  questionnaire  with  19  questions  and  a 
desk  audit/user  opinion  survey  with  38  questions.  The  sample  size  for  each  questionnaire  was 
eight  people  and  there  was  no  overall  analysis  of  the  results  (U.S.  Army  Medical  Department 
Board,  2003). 

The  AMEDD  board’s  assessment  focused  primarily  on  three  main  components:  effectiveness, 
suitability,  and  survivability  (U.S.  Army  Medical  Department  Board,  2003).  To  evaluate  a 
system’s  success,  one  must  keep  in  mind  the  setting,  objectives,  and  the  stakeholders  (Van  Der 
Meijden,  Tange,  Troost,  &  Hasman,  2003).  The  AMEDD  Board  essentially  addressed  one  of  the 
three  elements  of  success:  objectives.  Kaplan  (1997)  states  that  in  order  to  evaluate  the  impact 
of  medical  information  systems,  “research  should  be  designed  to  identify,  collect,  analyze,  and 
interpret  data  to  form  a  coherent  picture  of  processes  that  resulted  in  the  effects  or  impacts”  (p. 
95).  To  form  a  coherent  picture  of  the  Tactical  EMR,  it  is  also  critical  to  assess  the  success  of 
the  system  from  the  user  or  stakeholder’s  perspective. 

Information  system  success  is  difficult  to  define,  measure,  and  assess.  There  is  a  broad  range 
of  literature  that  both  suggests  the  best  way  to  measure  information  system  success  and  confuses 
the  issue.  In  an  attempt  to  reduce  the  confusion  and  organize  the  information  surrounding 
information  system  success,  DeLone  &  McLean  (1992)  proposed  six  categories,  which 
combined,  represent  a  simplified  model  for  information  success.  The  six  categories  are  1) 
system  quality,  (2)  information  quality,  (3)  use,  (4)  user  satisfaction,  (5)  individual  impact,  and 
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(6)  organizational  impact.  DeLone  &  McLean  describe  their  model  as  a  process  model  with  each 
of  the  individual  categories  having  an  interdependent  relationship.  System  quality  and 
information  quality  individually  and  simultaneously  have  an  impact  on  use  and  user  satisfaction. 
Use  and  user  satisfaction  have  a  direct  affect  upon  one  another.  They  in  turn  directly  affect 
individual  impact,  which  then  affects  organizational  impact.  To  effectively  measure  success, 
DeLone  &  McLean  suggest  that  individual  measures  from  the  different  categories  should  be 
combined  “to  create  a  comprehensive  measurement  instrument”  (DeLone  &  McLean,  1992,  p. 
88).  They  also  note  that  other  contingency  variables  should  be  included,  such  as  the  structure, 
size,  and  environment  of  the  organization  being  studied  (DeLone  &  McLean,  1992). 

Van  Der  Meijden,  Tange,  Troost,  and  Hasman  (2003)  conducted  a  thorough  study  of  both 
English  and  Dutch  literature  to  evaluate  the  framework  proposed  by  DeLone  &  McLean  (1992). 
Out  of  191  articles,  33  articles,  describing  29  different  information  systems,  were  selected  for 
analysis.  The  results  revealed  that  “information  quality  was  evaluated  in  64%  of  the  studies, 
system  quality  in  58%,  usage  in  36%,  user  satisfaction  in  48%,  individual  impact  in  45%,  and 
organizational  impact  in  39%”  (Van  Der  Meijden,  Tange,  Troost,  &  Hasman,  2003,  p.  237). 
Questionnaires,  chart  reviews,  and  interviews  were  the  primary  tools  used  for  assessing  system 
quality,  information  quality,  user  satisfaction,  individual  impact,  and  organizational  impact. 
Usage  was  assessed  through  time/work  sampling  and  content  analysis.  Key  attributes  of  each 
success  factor  were  also  identified.  Key  attributes  for  system  quality  were  ease  of  use,  response 
time,  and  time  savings.  Those  identified  for  information  quality  were  completeness,  accuracy  of 
data,  and  legibility.  Usage  attributes  were  number  of  entries,  frequency  of  use,  and  duration  of 
use.  The  attributes  noted  for  user  satisfaction  were  user  satisfaction  itself,  attitude,  and  user 
friendliness.  Individual  impact  attributes  were  change  of  work  patterns  and  direct  benefits. 
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Communication/collaboration  and  impact  on  patient  care  were  the  key  attributes  for 
organizational  impact  (Van  Der  Meijden,  Tange,  Troost,  &  Hasman,  2003). 

Van  Der  Meijden,  Tange,  Troost,  &  Hasman  (2003)  also  note  in  their  study  that  the  primary 
study  designs  used  to  evaluate  healthcare  information  systems  were  descriptive  and  correlational 
designs.  Kaplan  (1997)  adds  that  when  studying  medical  information  systems,  it  is  a  good  idea 
to  use  a  combination  of  quantitative  and  qualitative  methods.  Using  both  strengthens  the  ability 
to  form  a  coherent  picture  of  the  situation  under  study  (Kaplan,  1997). 

DeLone  &  McLean’s  framework  combined  with  the  key  attributes  identified  by  Van  Der 
Meijden,  Tange,  Troost,  &  Hasman  provide  a  good  format  to  evaluate  the  success  of  the  Western 
Regional  Tactical  EMR  from  the  user  perspective.  The  overall  framework  plus  the  individual 
attributes  together  facilitate  the  creation  of  a  survey  in  order  to  establish  the  “comprehensive 
measuring  instrument”  suggested  by  DeLone  &  McLean. 

Purpose 

The  purpose  of  the  study  was  to  evaluate  the  success  of  the  Tactical  EMR  and  its  ability  to 
improve  medical  documentation  from  the  user  perspective.  Using  DeLone  &  McLean’s  model 
and  Van  Der  Meijden,  Tange,  Troost,  &  Hasman’s  key  attributes,  a  survey  was  designed  to 
collect  user  feedback  regarding  the  Tactical  EMR  and  to  gain  critical  insight  into  specific  areas 
users  felt  needed  improvement.  The  survey  also  provided  user  recommendations  to  help 
leadership  improve  the  Tactical  EMR’s  development  and  future  implementation. 

The  variables  selected  in  this  study  to  broadly  define  success  were  information  quality, 
system  quality,  usage,  user  satisfaction,  individual  impact,  organizational  impact,  and 
demographics.  The  variable  selected,  more  specifically,  to  evaluate  the  success  of  the  Tactical 
EMR  and  its  overall  impact  on  the  organization  was  organizational  impact.  This  fits  with 


29 


Tactical  EMR  Assessment 


DeLone  &  McLean’s  flow  of  interdependent  variables,  in  which  organizational  impact  was  last. 
The  survey  questions  that  address  organizational  impact,  in  particular,  address  the  EMR’s  ability 
to  enhance  point-of-care  treatment,  establish  a  longitudinal  health  record,  maintain  continuity  of 
care,  improve  quality  of  care,  improve  medical  surveillance,  and,  ultimately,  provide  superior 
healthcare.  Demographics  were  included  to  incorporate  DeLone  &  McLean’s  recommendation 
of  adding  contingency  variables. 

The  dependent  variable  for  this  study  was  organizational  impact  (Y)  and  the  independent 
variables  were  system  quality  (Xi),  information  quality  (X2),  usage  (X3),  user  satisfaction  (X4), 
individual  impact  (X5),  unit  (X6),  rank  (X7),  duty  position  (Xg),  and  time  with  the  unit  (X9). 
System  quality  was  defined  by  the  variables  of  ease  of  use,  response  time,  and  time  savings. 
Information  quality  was  defined  by  the  variables  of  completeness,  accuracy  of  data,  and 
legibility.  Usage  was  defined  by  the  variables  of  number  of  entries,  frequency  of  use,  and 
duration  of  use.  User  satisfaction  was  defined  by  the  variables  of  user  satisfaction,  attitude,  and 
user  friendliness.  Individual  impact  was  defined  by  the  variables  of  change  in  work  patterns  and 
direct  benefits.  The  dependent  variable,  organizational  impact,  was  defined  by  the  variables  of 
communication/collaboration  and  impact  on  patient  care. 

Due  to  the  interdependent  relationship  presented  by  DeLone  &  McLean  and  their  assumption 
that  each  variable  will  have  some  affect  on  organizational  impact,  the  alternate  hypothesis 
selected  for  the  study  was  that  the  level  of  success  for  the  Tactical  EMR,  organizational  impact, 
will  vary  according  to  the  independent  variables.  The  null  hypothesis  was  that  the  level  of 
success  for  the  Tactical  EMR  would  not  vary  according  to  the  independent  variables. 
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Methods  and  Procedures 

The  research  method  chosen  for  this  project  was  a  cross-sectional,  descriptive  design  that 
was  primarily  quantitative  with  some  qualitative  analysis  (Cooper  &  Schindler,  2003).  A 
combination  of  both  types  of  analyses  helped  “improve  the  quality  of  results  through 
triangulation”  (Van  Der  Meijden,  Tange,  Troost  &  Hasman,  2003,  p.  242)  and  provided  a  more 
complete  analysis  of  the  Tactical  EMR.  Data  was  collected  using  a  nonprobability,  purposive 
sampling  approach  of  medical  personnel  assigned  to  the  3rd  Brigade,  2nd  Infantry  Division  and 
the  1st  Brigade,  25th  Infantry  Division  (Cooper  &  Schindler,  2003).  A  survey  instrument  was 
designed  using  DeLone  &  McLean’s  framework  for  information  system  success  and  the  key 
attributes  for  each  that  were  identified  by  Van  Der  Meijden,  Tange,  Troost,  &  Hasman. 

The  survey  was  designed  in  direct  consultation  with  the  WRMC  Chief  of  Staff  and  Chief 
of  Informatics.  Their  suggestions  and  guidance  helped  fine  tune  each  question  to  ensure  useful 
information  could  be  collected  to  evaluate  the  Tactical  EMR.  After  the  survey  design  was 
complete,  the  survey  was  given  to  the  Noncommissioned  Officer  in  Charge  (NCOIC)  of  the  3rd 
Brigade  Surgeon’s  office.  He,  in  addition  to  two  other  medic  noncommissioned  officers  (NCO), 
reviewed  the  cover  letter  and  survey  to  ensure  the  instructions  and  questions  were  readable, 
understandable,  and  clear.  Their  primary  concern  was  that  many  of  the  medics  who  used  the 
Tactical  EMR  used  it  sporadically  and  at  different  capacities.  To  ensure  that  respondents 
answered  the  surveys  with  consistency,  the  following  disclaimer  was  added  to  the  instructions, 
“When  completing  the  survey,  consider  the  components  of  the  Tactical  EMR  and  answer  the 
questions  according  to  your  experiences  using  the  medical  information  system  when  it  was 
available  and  working.”  The  final  survey  was  approved  by  the  WRMC  Chief  of  Staff. 
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The  survey  packet  consisted  of  a  cover  letter  explaining  the  purpose  of  the  study  and 
instructions  on  how  to  complete  the  survey  (Appendix  A).  The  survey  contained  29  questions 
(Appendix  B),  which  were  broken  down  into  three  sections.  Section  one  contained  four 
questions  to  collect  demographic  information.  Section  two  consisted  of  25  questions  designed  to 
evaluate  the  success  of  the  Tactical  EMR.  Each  question  was  ranked  using  a  Likert  scale  that 
measured  from  1  (strongly  disagree)  to  7  (strongly  agree).  Respondents  were  instructed  to 
answer  each  question  according  to  how  they  agreed  or  disagreed  to  each  given  statement. 

Survey  questions  5  through  28  were  grouped  according  to  each  variable  and  key  attribute  being 
evaluated  (Appendix  C).  Question  29  was  a  stand  alone  question  to  see  how  the  respondents  felt 
the  Tactical  EMR  was  progressing.  Section  three  was  for  any  comments  the  respondents  wanted 
to  include. 

Before  the  survey  was  distributed,  coordination  was  made  with  the  1st  Corp  Surgeon’s  office 
to  ensure  they  were  aware  that  the  study  was  being  conducted.  The  1st  Corp  Surgeon’s  office 
approved  the  study  and  offered  their  support.  The  primary  points  of  contact  for  each  unit  were 
the  perspective  Brigade  Surgeon  offices.  The  NCOIC  of  the  3rd  Brigade  Surgeon’s  office  and  an 
officer  from  the  1st  Brigade  Surgeon’s  office  served  as  the  central  distribution  and  collection 
points  for  the  surveys. 

Each  brigade  had  approximately  200  medical  personnel  assigned,  which  produced  a  sampling 
pool  of  approximately  400  personnel.  However,  personnel  turnover,  operational  tempo,  and 
training/mission  mission  requirements  affected  the  number  of  medical  personnel  available  to 
complete  the  surveys.  On  12  January  2005,  200  survey  packets,  which  included  the  survey  and 
cover  letter,  were  distributed  to  3rd  Brigade.  The  surveys  were  collected  and  returned  on  21 
January  2005.  Due  to  1st  Brigade’s  deployment  and  varying  locations  throughout  Iraq,  their 
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survey  packets  were  forwarded  electronically.  Survey  packets  for  1st  Brigade  were  reproduced 
locally  and  distributed  or  forwarded  by  email  to  survey  respondents  on  05  February  2005.  They 
were  collected  and  returned  on  1 1  February  2005.  The  combined  survey  response  rate  was  145 
out  of  400,  or  approximately  36%  (96  surveys  from  3rd  Brigade  and  49  from  1st  Brigade).  Of 
the  145  surveys  returned,  10  were  removed  because  survey  respondents  noted  in  the  comments 
section  that  they  had  never  used  the  medical  information  system.  The  remaining  sample  size  was 
N  =  135;  N  =  90  for  3rd  Brigade  and  N  =  45  for  1st  Brigade  (Table  1).  The  results  from  the 
survey  were  compiled  and  then  coded  for  statistical  analysis. 
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Table  1 


Demographic  Breakdown  of  Survey  Respondents 


Number  of 

Unit 

Responses 

Rank 

Duty  Position 

Time  with  Unit 

Physician 

0-12  12-24  24  + 

Enlisted  NCO  Officer  Unknown 

Medic  /Admin 

Months  Months  Months  Unknown 

3rd  Brigade  Total 

90 

51 

32 

6 

1 

84 

6 

7 

36 

47 

1/23  IN 

7 

3 

4 

0 

7 

0 

0 

3 

4 

2/3  IN 

15 

9 

3 

3 

12 

3 

1 

9 

5 

5/20  IN 

22 

14 

7 

1 

21 

1 

1 

7 

14 

1/14  CAV 

18 

10 

6 

1 

17 

1 

1 

9 

8 

296  BSMC 

17 

10 

6 

1 

16 

1 

1 

8 

8 

1/37  FA 

10 

5 

5 

0 

10 

0 

3 

0 

7 

18  EN 

1 

0 

1 

0 

1 

0 

0 

0 

1 

1st  Brigade  Total 

45 

14 

17 

12 

2 

33 

12 

12 

16 

16 

1 

1/25  IN  (unspecified) 

2 

0 

0 

2 

0 

2 

1 

1 

0 

1/5  IN 

5 

1 

3 

1 

4 

1 

0 

4 

1 

3/21  IN 

8 

2 

4 

2 

6 

2 

1 

4 

3 

1/24  IN 

6 

2 

1 

3 

3 

3 

1 

3 

2 

2/14  CAV 

5 

0 

3 

2 

3 

2 

1 

1 

3 

25  BSMC 

17 

8 

5 

2 

15 

2 

8 

3 

5 

2/8  FA 

2 

1 

1 

0 

2 

0 

0 

0 

2 

Total  Number  of  Responses 

135 

65 

49 

18 

3 

117 

18 

38 

104 

126 

1 

Note:  IN  =  Infantry;  CAV  =  Cavalry  Squadron;  BSMC  =  Brigade  Support  Medical  Company;  FA  =  Field  Artillery;  EN  =  Engineer 
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The  survey  results  provided  the  foundation  for  the  quantitative  analysis  and  the  comments 
section  provided  additional  insight  into  what  respondents  thought  about  the  Tactical  EMR.  The 
survey  comments,  coupled  with  the  information  presented  at  the  3/2  SBCT  Tactical  e-Medical 
Record  Research  and  Demonstration  Conference,  provided  the  foundation  for  the  qualitative 
analysis.  The  conference  was  the  first  major  forum  to  bring  the  Tactical  EMR’s  designers  and 
users  together  after  having  the  system  deployed  in  Iraq  for  a  year.  For  two  days,  a  series  of 
presentations  addressed  concepts,  implementation,  usage,  and  after  action  review  comments 
from  all  involved. 

Validity  and  Reliability 

Validity  refers  to  the  “extent  to  which  a  test  measures  what  we  actually  wish  to  measure” 
(Cooper  &  Schindler,  2003,  p.  231).  The  validity  of  the  survey  is  addressed  using  both  construct 
and  content  validity.  Construct  validity  examines  the  theory  and  the  measurement  tool  to 
determine  if  the  selected  variables  accurately  measure  the  construct  being  tested.  Content 
validity  assesses  the  extent  to  which  the  measurement  instrument  covers  the  topic  under  study 
(Cooper  &  Schindler,  2003).  In  this  study,  construct  validity  was  addressed  by  examining  how 
well  the  independent  variables  accurately  measured  success.  Content  validity  was  addressed  by 
how  well  the  key  factors  within  each  independent  variable  adequately  defined  each  variable. 
Both  the  construct  and  content  validity  of  the  survey  instrument  were  supported  by  the 
comprehensive  literature  review  conducted  by  Van  Der  Meijden,  Tange,  Troost,  &  Hasman 
(2003).  The  six  variables  chosen  for  the  study  were  validated  and  identified  by  them  as  the 
primary  indicators  of  success  in  29  different  infonnation  systems.  In  addition  to  the  six  factors, 
the  key  attributes  for  each  are  also  specified  and  identified  in  the  study.  Questions  selected  for 
the  survey  were  derived  from  both  the  key  attributes  and  survey  questions  used  during  the 
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AMEDD  Board  visit.  The  survey  was  reviewed  and  validated  by  the  WRMC  Chief  of  Staff, 
Chief  of  Informatics,  and  three  NCO  medics  assigned  to  3rd  Brigade  who  had  operational 
experience  with  the  Tactical  EMR.  The  review  and  feedback  from  these  personnel  further 
improved  the  construct  and  content  validity  of  the  survey  instrument. 

To  address  the  “accuracy  and  the  precision  of  the  measurement  procedure,”  reliability  must 
also  be  discussed  (Cooper  &  Schindler,  2003,  p.  231).  Reliability  in  this  study  was  addressed  by 
evaluating  the  internal  consistency  or  inter-item  reliability  of  the  questions  that  evaluated  each 
variable.  Evaluating  internal  consistency  assessed  the  consistency  of  the  questions  asked  within 
each  of  the  variables.  A  high  degree  of  correlation  would  indicate  high  reliability  of  the 
questions  chosen  to  measure  each  variable.  Cronbach’s  alpha,  “which  has  the  most  utility  for 
multi-items  scales  at  the  interval  level  of  measurement,”  was  used  to  test  the  inter-item  reliability 
(Cooper  &  Schindler,  2003,  p.  239).  According  to  the  UCLA  Academic  Technology  Services  “a 
reliability  coefficient  of  .80  or  higher  is  considered  as  "acceptable"  in  most  Social  Science 
applications”  (UCLA  Academic  Technology  Services,  n.d.).  All  of  the  items,  except  usage, 
which  resulted  in  an  alpha  =  .78,  exceeded  the  .80  threshold  indicating  an  “acceptable”  level  of 
reliability  (Table  2). 
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Table  2 


Cronbach ’s  Alpha  Reliability  Scores 


Variable 

Cronbach's  Alpha 

System  Quality 

Questions  5  through  8 

.89 

Information  Quality 

Questions  9  through  12  * 

.91 

Usage 

Questions  13  through  15 

.78 

User  Satisfaction 

Questions  16  through  21 

.90 

Individual  Impact 

Questions  22  through  24 

.91 

Organizational  Impact 

Questions  25  through  28 

.92 

Analytic  Methods 

Descriptive  statistics  were  calculated  to  provide  an  overview  of  the  characteristics  of  the  data. 
Mean  scores  and  standard  deviations  were  calculated  to  assess  how  survey  respondents  evaluated 
the  Tactical  EMR.  Questions  that  evaluated  a  particular  key  attribute  were  averaged  to  get  a 
mean  score  for  each.  The  mean  scores  of  the  key  attributes  were  then  combined  to  get  an  overall 
mean  for  the  variable  they  measured.  A  correlation  analysis  was  conducted  to  measure  the  linear 
association  between  the  variables.  A  Spearman  rho  correlation  analysis  was  selected  because  the 
assumption  of  normal  distribution  for  all  of  the  variables  was  not  met.  An  independent-samples  t 
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test  was  calculated  to  identify  significant  differences  between  the  units  and  then  the  duty 
positions  for  each  of  the  variables  in  the  model.  A  one  way  analysis  of  variance  (ANOVA)  was 
used  to  identify  significant  differences  between  the  rank  groups  and  then  time  with  the  unit  for 
each  of  the  variables  in  the  model.  If  significance  was  identified,  the  Fisher’s  least  significant 
difference  (LSD)  post-hoc  test  was  used  to  determine  where  differences  occurred  among  the 
groups.  Multiple  linear  regression  was  used  to  test  the  alternate  hypothesis  that  the  level  of 
success  will  vary  according  to  the  independent  variables.  The  results  of  the  regression  also 
provided  a  measure  as  to  whether  the  model  was  a  good  predictor  of  success  (Cronk,  2004; 
Cooper  &  Schindler,  2003). 

Results 

The  descriptive  statistics  for  the  variables  and  their  key  attributes  are  shown  in  Table  3  and 
the  descriptive  statistics  for  each  survey  question  are  shown  in  Table  4. 
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Table  3 


Descriptive  Statistics  for  Variables  and  Key  Attributes 


Variable 

n 

Mean 

Standard 

Deviation 

System  Quality 

135 

3.31 

1.38 

Ease  of  Use 

135 

3.00 

1.45 

Response  Time 

134 

3.33 

1.51 

Timesavings 

135 

3.57 

1.67 

Information  Quality 

135 

3.60 

1.40 

Completeness 

135 

3.43 

1.64 

Accuracy  of  Data 

135 

3.61 

1.40 

Legibility 

135 

3.75 

1.60 

Usage 

135 

2.98 

1.37 

Number  of  Entries 

134 

3.45 

1.62 

Frequency  of  Use 

135 

2.80 

1.71 

Duration  of  Use 

135 

2.68 

1.65 

User  Satisfaction 

135 

3.29 

1.35 

User  satisfaction 

135 

3.33 

1.33 

Attitude 

135 

3.27 

1.62 

User  Friendliness 

135 

3.27 

1.41 

Individual  Impact 

135 

3.04 

1.40 

Change  in  Work  Patterns 

135 

2.99 

1.43 

Direct  Benefits 

135 

3.10 

1.52 

Organizational  Impact 

135 

3.35 

1.47 

Communication/Collaboration 

134 

3.28 

1.65 

Impact  on  Patient  Care 

135 

3.39 

1.45 
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Table  4 


Descriptive  Statistics  for  Individual  Questions 


Variable 


Standard 
n  Mean  Deviation 


Question  5:  Using  the  SBCT  MIS  to  document  patient  care.. . 

Question  6:  Transferring  patient  encounters  between  system. . . 

Question  7:  The  SBCT  MIS  provides  the  information  I  need. . . 

Question  8:  The  SBCT  MIS  improves  my  ability  to  document. . . 

Question  9:  Medical  information  is  more  complete  because... 

Question  10:  Accuracy  of  medical  information  is  improved... 

Question  1 1 :  The  SBCT  MIS  provides  enough  information. . . 

Question  12:  Information  in  the  SBCT  MIS  is  easy  to  read... 

Question  13:  The  SBCT  MIS  has  enabled  me  to  create... 

Question  14:  I  use  the  SBCT  MIS  every  time  I  interact  with.. . 

Question  15:  Documenting  medical  information  in  the  SBCT... 

Question  16:  The  SBCT  MIS  improves  medical  documentation... 

Question  17:  The  SBCT  MIS  improves  medical  documentation... 

Question  1 8:  The  SBCT  MIS  is  an  improvement  over  paper. . . 

Question  19:  The  SBCT  MIS  is  worth  the  time  and  effort... 

Question  20:  Medical  Information  stored  in  the  SBCT  MIS... 

Question  21:  I  use  the  SBCT  MIS  more  often  because... 

Question  22:  The  SBCT  MIS  has  made  my  job/mission  easier... 

Question  23:  The  SBCT  MIS  has  improved  my  ability  to... 

Question  24:  The  increased  availability  of  medical  information. . . 

Question  25:  The  SBCT  MIS  has  improved  the  communication... 

Question  26:  Access  to  more  medical  information  has  improved. . . 

Question  27:  Using  the  SBCT  MIS  has  increased  medical... 

Question  28:  The  increase  in  medical  documentation  has  had... 

Question  29:  I  believe  the  implementation  of  the  SBCT  MIS... _ 

Note:  SBCT  MIS  =  Stryker  Brigade  Combat  Team  Medical  Information  System 


134 

2.95 

1.59 

135 

3.06 

1.52 

134 

3.33 

1.51 

135 

3.57 

1.67 

135 

3.43 

1.64 

134 

3.51 

1.55 

135 

3.71 

1.48 

135 

3.75 

1.60 

134 

3.45 

1.62 

135 

2.80 

1.71 

135 

2.68 

1,65 

135 

2.43 

1.56 

134 

4.08 

1.69 

134 

3.49 

1.56 

135 

3.27 

1.62 

135 

3.67 

1.69 

133 

2.83 

1.45 

135 

2.95 

1.50 

134 

3.04 

1.54 

135 

3.10 

1.52 

134 

3.28 

1.65 

133 

3.55 

1.62 

134 

3.24 

1.52 

135 

3.38 

1.59 

135 

3.79 

1.72 
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As  noted  earlier,  survey  respondents  were  instructed  to  score  questions  on  Likert  Scale  of  1 
(strongly  disagree)  to  7  (strongly  agree)  for  given  statements.  One  the  whole,  all  variables  were 
ranked  below  4  (neither  agree  nor  disagree)  on  the  Likert  scale.  Of  the  five  variables  selected  to 
evaluate  the  success  of  the  Tactical  EMR  and  its  ability  to  improve  medical  documentation, 
respondents  ranked  information  quality  the  highest  (m  =  3.60,  sd  =  1.40)  and  usage  the  lowest  (m 
=  2.98,  sd  =  1 .37).  Of  the  key  attributes,  legibility  (m  =  3.75,  sd=  1 .60)  ranked  the  highest 
overall  key  attribute  and  duration  of  use  (m  =  2.68,  sd  =  1 .65)  the  lowest.  The  question  with  the 
highest  mean  score  was  question  17  (m  =  4.08,  sd  =  1.69),  the  SBCT  medical  information  system 
improves  medical  documentation  in  fixed  medical  treatment  facilities.  The  question  with  the 
lowest  mean  score  was  question  18  (m  =  2.43,  sd  =  1.56),  the  SBCT  medical  information  system 
improved  medical  documentation  under  stressful,  fast  paced  battlefield  conditions.  Question  29 
(m  =  3.79,  sd  =  1.72),  assessed  whether  users  felt  the  Tactical  EMR  was  progressing  in  the  right 
direction.  Appendix  D  contains  a  complete  breakdown  of  descriptive  statistics  for  each  unit  and 
Appendix  E  contains  the  rankings  of  each  variable  and  key  attribute  by  demographic  category. 

A  Spearman  rho  correlation  was  calculated  for  each  of  the  variables  (Appendix  F).  A 
significant  positive  correlation  (p  <  .001)  was  found  between  the  variables  of  system  quality, 
information  quality,  usage,  user  friendliness,  individual  impact,  and  organizational  impact. 

There  was  a  significant  negative  correlation  between  time  with  the  unit  and  information  quality, 
usage,  and  organization  impact  at  p  <  .001  and  between  time  with  unit  and  user  satisfaction  and 
individual  impact  at  p  <  .05. 

The  independent-samples  t  test  was  used  to  compare  the  mean  scores  by  unit  and  duty 
position  for  each  variable.  No  significant  differences  were  found  between  the  mean  scores  of  3rd 
Brigade  and  1st  Brigade  or  the  mean  scores  of  physician/admin  and  medic  for  each  variable 
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(Appendix  G).  The  ANOVA  was  used  to  compare  the  mean  scores  for  the  different  rank  groups 
and  time  with  unit  categories  for  each  variable.  The  mean  score  of  each  variable  was  compared 
with  the  mean  scores  of  the  three  different  rank  groups  (enlisted,  NCO,  and  officer)  and  no 
significant  difference  was  found  between  the  three  rank  groups  (Appendix  H).  When  the  mean 
score  of  each  variable  was  compared  with  mean  scores  of  each  time  group,  a  significant 
difference  was  found  among  time  groups  for  information  quality,  usage,  user  satisfaction,  and 
organizational  impact  (Table  5). 
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Table  5 


Analysis  of  Variance  for  Time  with  Unit 


Variable 

Sum  of  Squares  Mean  Square 

F 

Sig. 

System  Quality 

Between  Groups 

8.48 

4.24 

2.29 

.11 

Within  Groups 

242.95 

1.86 

Total 

251.43 

Information  Quality 

Between  Groups 

18.27 

9.14 

4.94 

.01* 

Within  Groups 

242.19 

1.85 

Total 

260.46 

Usage 

Between  Groups 

29.92 

14.96 

8.98 

.00* 

Within  Groups 

218.18  ' 

1.67 

Total 

248.10 

User  Satisfaction 

Between  Groups 

11.25 

5.63 

3.21 

.04* 

Within  Groups 

229.91 

1.76 

' 

Total 

241.16 

Individual  Impact 

Between  Groups 

8.68 

4.34 

2.23 

.11 

Within  Groups 

254.72 

1.94 

Total 

263.40 

Organizational  Impact 

Between  Groups 

25.68 

12.84 

6.41 

.00* 

Within  Groups 

262.61 

2.01 

Total 

288.29 

Note:  df=  2,129;  *p  <  .05 
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Fisher’s  LSD  post-hoc  test  was  used  to  examine  the  differences  between  time  groups.  The 
analysis  revealed  that  individuals  assigned  to  the  unit  from  0  to  12  months  scored  information 
quality,  usage,  user  satisfaction,  and  organizational  impact  significantly  higher  than  those 
assigned  12  to  24  months  and  24  or  more  months  (Table  6). 
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Table  6 


Fisher's  Least  Significance  Difference  Post  Hoc  Test  for  Time  with  Unit 


Variable 

Category  1 

Category  2 

Mean 

Difference 

Std.  Error 

Sig. 

95%  Confidence  Interval 
Lower  Upper 

Bound  Bound 

Information  Quality 

0-12  Months 

12-24  Months 

.75* 

.36 

.04 

.03 

1.47 

0-12  Months 

24  +  Months 

1.11* 

.36 

.00 

.40 

1.81 

12-24  Months 

0-12  Months 

-.75* 

.36 

.04 

-1.47 

-.03 

12-24  Months 

24  +  Months 

.36 

.26 

.16 

-.15 

.86 

24  +  Months 

0-12  Months 

-1.11* 

.36 

.00 

-1.81 

-.40 

24  +  Months 

1 2-24  Months 

-.36 

.26 

.16 

-.86 

.15 

Usage 

0-12  Months 

12-24  Months 

1.09* 

.35 

.00 

.40 

1.77 

0-12  Months 

24  +  Months 

1.43* 

.34 

.00 

.76 

2.10 

12-24  Months 

0-12  Months 

-1.09* 

.35 

.00 

-1.77 

-.40 

12-24  Months 

24  +  Months 

.34 

.24 

.16 

-.13 

.82 

24  +  Months 

0-12  Months 

-1.43* 

.34 

.00 

-2.10 

-.76 

24  +  Months 

12-24  Months 

-.34 

.24 

.16 

-.82 

.13 

User  Satisfaction 

0-12  Months 

12-24  Months 

.60 

.36 

.10 

-.10 

1.30 

0-12  Months 

24  +  Months 

.87* 

.35 

.01 

.19 

1.56 

12-24  Months 

0-12  Months 

-.60 

.36 

.22 

-1.44 

.24 

12-24  Months 

24  +  Months 

.27 

.25 

.27 

-.22 

.76 

24  +  Months 

0-12  Months 

-.87* 

.35 

.04 

-1.69 

-.05 

24  +  Months 

12-24  Months 

-.27 

.25 

.51 

-.86 

.31 

Organizational  Impact 

0- 1 2  Months 

1 2-24  Months 

1.05* 

.38 

.01 

.30 

1.80 

0-12  Months 

24  +  Months 

1.33* 

.37 

.00 

.59 

2.06 

12-24  Months 

0- 1 2  Months 

-1.05* 

.38 

.02 

-1.95 

-.15 

1 2-24  Months 

24  +  Months 

.28 

.27 

.30 

-.25 

.80 

24  +  Months 

0-12  Months 

-1.33* 

.37 

.00 

-2.20 

-.45 

24  +  Months 

12-24  Months 

-.28 

.27 

.55 

-.90 

.35 

Note:  p  <  .05 
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Of  particular  note  was  the  significant  difference  between  the  time  groups  and  the  dependent 
variable  representing  success,  or  organizational  impact.  Individuals  assigned  to  the  unit  0  to  12 
months  (m  =  4.38,  sd=  1.33)  ranked  success  significantly  higher  than  those  assigned  12  to  24 
months  ( m  =  3.33,  sd  =  1.25)  and  24  months  or  more  (m  =  3.05,  sd=  1.56).  There  was  no 
significant  difference  between  the  means  scores  of  those  assigned  12  to  24  months  and  24  or 
more  months  for  any  of  the  variables.  On  explanation  for  this  may  be  that  users  assigned  to  the 
units  longer  had  more  institutional  knowledge  and  exposure  to  the  Tactical  EMR  as  it  was  being 
introduced,  fixed,  and  upgraded.  Their  experiences  with  the  system  may  have  varied  more  than 
newer  users  who  had  a  more  limited  frame  of  reference  for  comparison  and  evaluation. 

Multiple  linear  regression  was  used  to  test  the  alternate  hypothesis  that  the  level  success 
(organizational  impact)  will  vary  according  to  the  independent  variables.  The  results  of  the 
regression  were  significant  (F( 9,  122)  =  39.22,  p  <  .001),  with  an  R2  of  .74  (Table  7). 

Table  7 

Regression  Analysis  for  Organizational  Impact  as  a  Function  of  the 
Independent  Variables  and  as  a  Function  of  the  Individual  Questions _ 


Adjusted 


Model 

R 

R  Square 

R  Square 

SEE 

F 

Sig. 

Independent  Variables* 

.86 

.74 

.72 

.77 

39.22 

.00 

Individual  Questions** 

.91 

.83 

.78 

.70 

18.72 

.00 

Note:  *df  =  9,  122;  p  <  .001  /  **df  =  25,  99;  p  <  .001 

The  alternate  hypothesis  was  accepted  confirming  that  the  level  of  success  did  vary  according  to 
the  independent  variables.  Of  the  nine  independent  variables,  individual  impact  was  found  to  be 
the  only  significant  predictor  for  organizational  impact.  The  key  attributes  that  pertain  to 
individual  impact  were  change  in  work  patterns  and  direct  benefits.  As  a  model,  the  independent 
variables  accounted  for  approximately  74%  of  the  variance  in  organizational  impact,  which 
indicated  that  it  was  a  relatively  strong  model  to  predict  success. 
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A  second  multiple  linear  regression  test  was  performed  to  determine  if  there  were  any 
particular  questions  that  were  significant  predictors  of  organizational  impact.  The  variables  were 
replaced  with  questions  and  the  demographic  variables  remained  the  same.  The  results  of  the 
second  regression  were  also  significant  with  F( 25,  99)  =  18.72,  p  <  .001),  with  an  R2  of  .83 
(Table  7).  Ranked  from  strongest  to  weakest,  questions  8,  23,  24,  21  (negative  t  value),  17,  and 

20  were  found  to  be  the  significant  predictors  for  organizational  impact.  Question  8  was  a 
measure  of  timeliness  and  assessed  if  the  SBCT  medical  information  system  improved  the  user’s 
ability  to  document  medical  information.  Question  23  was  a  measure  of  change  in  work  patterns 
and  assessed  whether  the  SBCT  had  improved  the  users  ability  to  provide  better  medical  care. 
Question  24  was  the  only  measure  of  direct  benefits  and  assessed  whether  the  increased 
availability  to  medical  information  had  improved  the  user’s  overall  performance  level.  Question 

21  was  a  measure  of  user  friendliness  and  assessed  whether  the  SCBT  medical  information 
system  was  used  more  often  because  it  was  user  friendly.  Question  17  was  a  measure  of  user 
satisfaction  and  assessed  if  the  SBCT  medical  information  system  improved  medical 
documentation  in  fixed  medical  treatment  facilities.  The  last  question,  question  20,  assessed  if 
medical  information  in  the  SBCT  medical  information  system  was  easier  to  retrieve  than 
information  stored  in  paper-based  records. 

Discussion 

When  reviewing  the  results,  it  is  important  to  keep  in  mind  that  the  Tactical  EMR  is  still  in  its 
early  stages  of  development  and  implementation.  The  concept  for  the  Tactical  EMR  was  the 
same  for  both  SBCTs,  however,  operational  constraints  affected  how  each  unit  used  the  Tactical 
EMR  and  to  what  capacity.  On  the  whole,  the  survey  respondents  evaluated  the  success  of  the 
Tactical  EMR  and  its  ability  to  improve  medical  documentation  relatively  low.  The  mean  score 


47 


Tactical  EMR  Assessment 


for  all  five  variables  fell  below  4  on  the  7-point  Likert  scale.  The  scores,  however,  should  not  be 
overshadowed  by  the  fact  that  this  provides  an  excellent  opportunity  to  examine  critical  areas 
that  users  felt  needed  improvement  and  to  discuss  confounding  factors  that  influenced  both  the 
survey  results  and  the  successful  implementation  of  the  Tactical  EMR. 

Information  quality,  which  sought  to  evaluate  the  completeness,  accuracy,  and  legibility  of 
medical  information,  was  ranked  highest  among  the  variables.  Its  key  attributes  of  legibility  and 
accuracy  of  data  ranked  the  highest  among  all  of  the  respective  key  attributes  evaluated  in  the 
study.  This  indicated  that  users  felt  that  information  within  the  Tactical  EMR  was  easy  to  read 
and  understand,  that  the  Tactical  EMR  had  improved  the  accuracy  of  medical  information,  and 
that  the  Tactical  EMR  provided  enough  information  to  provide  quality  patient  care. 

Organizational  impact,  the  second  highest  variable,  evaluated  what  impact  users  felt  the 
Tactical  EMR  had  on  the  organization.  The  two  areas  of  evaluation  were  communication  and 
collaboration  and  impact  on  patient  care.  These  attributes  measured  whether  the  Tactical  EMR 
had  improved  communication  of  medical  information  from  the  point  of  care  to  higher  levels  of 
care,  if  access  to  more  information  had  improved  continuity  of  care,  if  using  the  Tactical  EMR 
had  increased  medical  documentation,  and  if  the  increase  in  medical  documentation  had  had  a 
positive  impact  on  patient  care.  Although  close,  users  scored  impact  on  patient  care  slightly 
higher  than  communication  and  collaboration. 

System  quality,  the  third  highest  variable,  evaluated  the  Tactical  EMR’s  ease  of  use,  response 
time,  and  time  savings.  Time  savings  was  ranked  the  third  highest  overall  key  attribute,  which 
meant  that  users  felt  the  that  the  Tactical  EMR  had,  to  some  degree,  improved  their  ability  to 
document  medical  information.  Ease  of  use,  on  the  other  hand,  was  ranked  among  the  five 
lowest  key  attributes.  This  indicated  that  users  did  not  necessarily  feel  that  using  the  Tactical 
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EMR  to  document  patient  care  was  easier  than  previous  methods  of  documentation.  It  also 
indicated  that  users  did  not  necessarily  feel  that  transferring  patient  encounters  between  system 
components  was  easy.  Ease  of  use  did  not  delineate  between  use  of  the  Tactical  EMR  on  the 
battlefield  or  in  fixed  medical  facilities.  As  will  be  discussed  later,  medics  preferred  using  field 
medical  cards  rather  than  the  PIC  and  BMIS-T  because  it  was  quicker,  easier,  and  there  were 
some  interface  issues.  The  IOM  reports  highlight  the  importance  of  using  information 
technology  to  improve  patient  care.  Better  access  to  medical  information  or  past  medical  history 
can  improve  overall  quality  of  care.  Documenting  medical  care  may  not  always  be  easy,  but 
when  done  properly,  the  availability  of  information  can  impact  future  medical  care.  It  is 
important  to  educate  Tactical  EMR  users  that  until  easier  methods  of  documentation  are 
developed,  it  is  still  vital  that  patient  care  be  documented  in  a  manner  that  will  provide  some  sort 
of  longitudinal  patient  care  history.  When  and  where  to  document  the  medical  care  will  be 
operationally  dependant  and  will  vary  according  to  each  user. 

User  Satisfaction,  the  fourth  highest  variable,  sought  to  evaluate  user  satisfaction,  user 
friendliness,  and  attitude  toward  the  Tactical  EMR.  User  satisfaction  contained  the  highest  and 
lowest  ranked  questions  for  the  survey.  A  common  sentiment  among  users  from  both  brigades 
was  that  the  BMIS-T  device  was  not  suitable  for  combat  or  trauma  situations,  it  was  time 
consuming,  non-tactical,  fragile,  and  cumbersome.  The  interface  with  the  PIC  was  also 
questionable.  This  was  confirmed  when  users  scored  the  question,  “the  SBCT  medical 
information  system  improves  medical  documentation  under  stressful,  fast-paced  battlefield 
conditions”  the  lowest.  As  for  using  the  Tactical  EMR  in  garrison,  users  scored  the  question, 
“the  SBCT  medical  information  system  improves  medical  documentation  in  fixed  treatment 
facilities”  the  highest.  Users  noted  that  the  field  medical  card  remained  the  easiest  way  to 
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transfer  information  back  to  the  battalion  aid  station  and  that  the  most  appropriate  use  or  place 
for  the  Tactical  EMR  was  at  the  battalion  aid  station  level  or  higher. 

As  shown  by  the  results  of  the  multiple  linear  regression  analysis,  individual  impact,  the  fifth 
highest  variable,  was  found  to  have  the  greatest  influence  on  organizational  impact,  or  success. 
The  better  the  medics  are  able  to  perform,  the  better  the  outcome  for  the  organization.  Individual 
impact  evaluated  change  in  work  patterns  and  direct  benefits.  Both  ranked  in  the  bottom  five 
key  attributes,  indicating  that  the  majority  of  users  felt  that  the  Tactical  EMR  did  not  make  their 
job  easier  nor  did  it  improve  their  ability  to  provide  better  medical  care.  They  also  felt  that  the 
increase  in  medical  information  did  not  necessarily  improve  their  overall  performance  level. 

This  is  important  to  note  because,  on  average,  despite  the  Tactical  EMR’s  ability  to  provide 
enough  information  for  quality  care  (accuracy  of  data);  it  did  not  necessarily  improve  some 
user’s  perceived  productivity  or  efficiency  levels.  There  were  a  select  few,  however,  who 
consistently  used  the  Tactical  EMR  and  experienced  increased  productivity  and  efficiency  levels. 
One  example  is  with  the  SBCT’s  imbedded  physical  therapist.  Despite  difficulties  with  the 
Tactical  EMR,  the  physical  therapist  religiously  used  the  system,  which  ultimately  allowed  him 
to  enhance  his  ability  to  provide  in-theater  care.  Access  to  the  ICDB’s  longitudinal  patient 
record  enabled  the  physical  therapist  to  prevent  unnecessary  medical  evacuations  back  to  the 
United  States  saving  the  unit  over  1000  man  days  during  their  one  year  deployment.  As  shown 
by  the  physical  therapist,  ensuring  the  Tactical  EMR  has  some  value-added  benefit  for  the  user  is 
important  for  increasing  usage  and  improving  medical  documentation.  With  few  visible  benefits 
or  an  unclear  purpose,  users  may  choose  not  to  continue  using  the  Tactical  EMR. 

The  final  variable,  usage,  was  ranked  the  lowest  of  the  six  variables.  Usage  evaluated 
number  of  entries,  frequency  of  use,  and  duration  of  use.  Despite  the  users’  feelings  that  the 
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Tactical  EMR  enabled  them  to  create  and  store  more  patient  encounters  (number  of  entries),  they 
did  not  use  it  every  time  they  interacted  with  patients.  Users  also  felt  that  documenting  medical 
information  with  the  Tactical  EMR  was  not  necessarily  quicker  than  documenting  information  in 
patient  records.  Frequency  of  use  and  duration  of  use  were  ranked  the  lowest  overall  key 
attributes  out  of  the  sixteen.  The  reasons  for  not  using  the  Tactical  EMR  varied  and  like  the 
other  variables  were  influenced  by  confounding  factors.  Important  confounding  factors  were 
identified  in  the  comments  section  of  the  survey  and  during  the  3/2  SBCT  Tactical  e-Medical 
Record  Research  and  Demonstration  Conference.  Three  commonly  noted  factors  that  influenced 
the  success  of  Tactical  EMR  and  its  ability  to  improve  medical  documentation  were  the 
functioning  of  the  individual  system  components,  connectivity  issues,  and  user  training. 

The  first  system  component  was  the  PIC,  which  users  from  both  units  found  to  be  unreliable. 
They  were  unusable  when  dirty  and  they  corroded  easily.  Users  also  noted  that  not  all  of  units 
they  supported  used  the  PIC.  One  important  c'omment  and  follow-on  discussion  centered  on  the 
exact  role  of  the  PIC.  Users  in  3rd  Brigade  were  unsure  if  the  PIC’s  primary  purpose  was  for 
recording  immunizations,  sick  call  visits,  trauma  care,  or  all  of  the  above. 

The  second  component  was  BMIS-T.  The  most  resounding  comment  regarding  BMIS-T  was 
that  it  was  impractical  in  combat  and  trauma  situations.  Users  felt  that  BMIS-T  was  too  fragile, 
cumbersome,  non-tactical,  and  limited  in  battery  power.  Once  the  system’s  memory  became 
full,  the  system  lagged.  The  interface  with  the  PIC  did  not  always  work  and  it  took  too  long  for 
the  system  to  start  up.  Users  also  indicated  that  it  was  hard  to  download  information  to  the 
computer,  but  did  not  specify  if  it  was  CHCS  II-T  or  ICDB.  There  were  mixed  reviews  about 
whether  there  were  too  many  or  too  few  drop  down  menus.  There  were  also  some  questions 
about  when  to  use  BMIS-T,  at  the  point  of  injury  or  afterwards.  One  user  commented,  “Amidst 
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battlefield  conditions,  I  didn’t  think  of  grabbing  my  system.  All  I  thought  about  was  performing 
my  duties  as  medic.”  Ground  transport  times  in  Iraq  were  also  quick  and  did  not  facilitate  good 
documentation.  Therefore,  users  commented  that  the  field  medical  card  remained  the  easiest 
form  of  medical  documentation.  Although  users  felt  that  the  field  medical  card  was  the  easiest 
way  to  document  medical  patient  care,  it  is  not  necessarily  the  most  accurate  or  durable  way  to 
record  patient  care.  Field  medical  cards  are  limited  in  the  medical  information  they  can  provide 
and  they  do  not  often  make  it  through  the  different  levels  of  medical  care.  One  important  medic 
duty  is  documenting  medical  care.  Therefore,  the  key  is  finding  a  transparent,  yet  satisfactory, 
medium  between  the  most  accurate,  efficient  way  to  document  battlefield  medical  care  and  the 
most  durable  method  for  doing  so.  Users  did  indicate  that  BMIS-T  was  good  for  sick  call  and 
routine  care.  Sick  call  and  routine  care  afforded  more  time  to  input  data  and  work  with  the 
system. 

The  third  system  component  was  CHCS  II-T.  CHCS  II-T  is  currently  deployed  in  its  most 
rudimentary  form  and  all  of  the  features  for  the  Army’s  proposed  Tactical  EMR  are  not  fully 
operational.  According  to  the  3rd  Brigade  Surgeon,  roughly  50%  of  the  people  used  CHCS  ITT 
and  50%  used  ICDB.  When  CHCS  II-T  was  used,  it  was  used  primarily  as  a  decentralized 
database  and  the  data  could  only  travel  as  far  as  the  laptops  went.  1st  Brigade  is  currently 
deployed  and  using  CHCS  II-T.  However,  they  have  had  some  difficulty  keeping  it  functioning. 
If  users  did  not  use  CHCS  II-T  often,  inputting  data  became  difficult  and  time  consuming.  One 
physician  noted  that  accessing  notes  in  CHCS  II-T  was  difficult  because  the  notes  were  behind 
too  many  “non-intuitive”  click  menus. 

The  fourth  component  was  ICDB.  ICDB  streamlines  the  flow  of  information  and  is  the  crux 
of  the  Western  Regional  Tactical  EMR.  Users  of  ICDB  found  it  to  be  less  problematic,  easier  to 
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use,  and  more  flexible.  Users  were  able  to  access  past  medical  histories,  and  create,  save,  and 
manipulate  patient  notes  when  needed.  Not  only  did  it  provide  reach-back  capability,  which 
allowed  physicians  in  Iraq  to  review  notes  from  MAMC,  it  allowed  MAMC  physicians  to  review 
notes  created  in  Iraq.  The  users  were  pleased  with  ICDB  and  the  automation  support  they 
received  to  maintain  the  Western  Regional  Tactical  EMR’s  functionality.  A  major  drawback  for 
ICDB  was  its  requirement  for  internet  access. 

One  of  the  primary  elements  crucial  to  the  use  of  ICDB  and  the  success  of  the  Tactical  EMR 
was  connectivity.  At  all  levels  of  care,  the  operational  environment  dictated  the  availability  of 
resources  and  units  had  intermittent  internet  connectivity.  As  is  traditional  with  many  combat 
units,  the  emphasis  in  the  battlefield  is  the  war- fighting  effort  and  those  systems  that  enhance  it. 
Internet  connectivity  for  the  SBCT  medical  personnel  was  not  necessarily  a  unit  priority.  This 
further  reaffirmed  the  users’  claims  that  the  Tactical  EMR  worked  well  in  fixed  medical 
treatment  facilities.  Command  support  and  leadership  awareness  of  the  capabilities  of  a  fully 
functional  Tactical  EMR  are  the  keys  to  ensuring  medical  personnel  get  the  resources  they  need. 
Another  important  feature  of  connectivity  was  how  the  Tactical  EMR  was  deployed.  The 
SBCTs  were  the  first  of  the  conventional  forces  to  receive  a  Tactical  EMR.  Besides  improving 
the  medical  documentation  for  their  Soldiers,  users  were  limited  with  what  they  could  do  with 
the  electronic  information  once  captured  and  stored.  For  example,  it  was  not  guaranteed  that 
medical  providers  above  the  Stryker  Brigade  Support  Medical  Company  would  have  the 
capability  to  read  the  PIC  or  access  updated  information  from  CHCS  II-T  or  ICDB.  The  result 
was  duplication  in  work  that  required  users  to  input  everything  electronically  and  then  complete 
paper  copies  to  forward  through  the  higher  echelons  of  medical  care.  Some  users  commented 
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that  they  knew  the  medical  information  would  not  go  anywhere  so  they  saw  no  reason  to 
continue  using  the  Tactical  EMR. 

Training  was  another  important  factor  that  influenced  the  success  of  the  Tactical  EMR.  Not 
only  was  more  training  needed,  but  continual  training  was  needed.  Survey  respondents  from  3rd 
Brigade  felt  they  received  little  to  no  training  on  the  how  to  use  the  Tactical  EMR.  This  was  the 
result  of  both  pre-deployment  training  requirements  and  the  rapid  fielding  of  the  system.  To 
supplement  pre-deployment  training,  users  received  small  amounts  of  training  while  in  Iraq  and 
their  natural  curiosity  led  them  to  experiment  with  the  system.  However,  neither  was  sufficient 
for  sustained  use  of  the  Tactical  EMR.  Survey  respondents  from  1st  Brigade  had  more  time  to 
train  on  the  equipment,  particularly  with  the  ICDB  data  flow.  Comments  favored  ICDB  over 
CHCS  II-T,  but  comments  regarding  the  PIC  and  BMIS-T  were  similar.  Both  units  appeared  to 
be  slightly  confused  over  which  system  was  the  right  system  to  use,  CHCS  II-T  or  ICDB.  The 
lack  of  training  or  emphasis  on  the  differences  or  the  importance  of  the  Western  Regional 
Tactical  EMR  versus  the  Army’s  proposed  architecture  added  to  the  user’s  confusion  and 
frustration  with  using  the  Tactical  EMR. 

In  this  study,  it  was  difficult  to  determine  the  degree  to  which  the  confounding  factors 
affected  the  users.  However,  it  is  important  to  note  that  the  confounding  factors  did  play,  and 
will  continue  to  play,  an  important  role  in  evaluating  the  success  of  the  Tactical  EMR. 
Controlling  for  and  reducing  the  affects  of  these  confounders  will  ultimately  improve  the  success 
of  the  Tactical  EMR.  Overall,  the  results  suggest  that  the  Tactical  EMR  has  the  ability  to 
improve  medical  documentation  (information  quality),  but  additional  emphasis  needs  to  be 
placed  on  elements  that  improve  individual  impact  and  usage.  The  Western  Regional  Tactical 
EMR  appeared  to  be  favored  over  the  Army’s  current  proposed  architecture,  especially  with  1st 
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Brigade.  It  is  currently  more  flexible,  user  friendly,  allows  for  real-time  updates,  and  provides 
reach-back  capability  when  internet  access  is  available.  Some  of  its  functionality  is  important  to 
keep  in  mind  as  the  Army  continues  to  develop  the  CHCS  II-T  Tactical  EMR.  The  Western 
Regional  Tactical  EMR  is  an  important  step  in  the  development  of  a  Tactical  EMR  for  the 
SBCTs  and  it  is  a  reliable  interim  solution  until  the  Army’s  proposed  EMR  architecture  is 
complete  and  fully  operational. 

Recommendations 

The  purpose  of  the  study  was  to  evaluate  the  success  of  the  Western  Regional  Tactical  EMR 
and  its  ability  to  improve  medical  documentation  from  the  user  perspective.  Survey  results 
provided  important  feedback  and  provided  critical  insight  into  specific  areas  users  felt  needed 
improvement.  Early  identification  of  the  Tactical  EMR’s  strengths  and  weaknesses  allows 
leadership  to  correct  problems  in  a  timely  manner,  provides  valuable  insight  for  implementation 
with  other  units,  and  ensures  Soldiers  continue  to  receive  superior  medical  care. 

As  the  Tactical  EMR  continues  to  be  developed  and  improved,  there  are  a  few  important 
things  to  keep  in  mind.  In  the  article  “Evaluating  Clinical  Information  Systems,”  Kirkley  & 
Rewick  (2003)  state  that  a  successful  clinical  information  system  should  think  like  the  user 
thinks.  The  Tactical  EMR  is  a  variant  to  the  traditional  clinical  information  system,  but  the 
concept  is  the  same.  In  order  for  the  Tactical  EMR  (both  the  Western  Regional  and  Army 
version)  to  think  like  the  user  and  promote  usage,  user  needs  and  the  workflow  processes  must 
be  understood  at  all  levels.  Once  identified,  both  must  be  matched  with  the  system  components 
to  ensure  the  link  is  supported.  “Simply  automating  existing  paper-based  processes  does  not 
allow  the  organization  to  shed  inefficiencies”  (Kirkley  &  Rewick,  2003,  p.  646). 
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As  identified  by  the  results,  the  variable  ranked  lowest  for  the  Tactical  EMR  was  usage. 
According  to  Kirkley  &  Rewick  (2003),  consistency  of  design  and  simplicity  are  two  keys  to 
improving  usability  or  usage.  Information  must  be  presented  logically  and  interfaces  must  be 
both  intuitive  and  transparent  to  the  user.  “Non-intuitive”  or  inefficient  interfaces,  as  were  noted 
about  the  PIC,  BMIS-T,  and  CHCS  II-T,  can  lead  to  dissatisfaction  and  non-use  of  the  system. 
Systems  that  are  complex  can  detract  from  the  mission  at  hand.  In  the  case  of  the  Tactical  EMR, 
the  mission  involves  both  patient  care  and  medical  documentation.  “A  system  not  designed  with 
usability  principles  in  mind  will,  at  best,  not  be  used  and,  at  worst,  will  interfere  with  workflow” 
(Kirkley  &  Rewick,  2003,  647). 

The  3/2  SBCT  Tactical  e-Medical  Record  Research  and  Demonstration  Conference  was  a 
great  step  towards  better  understanding  user  needs  and  workflow  processes.  It  is  important  that 
the  information  gathered  at  the  conference  not  be  forgotten  and  that  it  is  applied  to  the  next 
generation  of  Tactical  EMR  components.  When  Soldiers  are  given  a  new  piece  of  equipment 
and  are  told  to  make  it  work,  they  will  do  their  best.  However,  they  rely  upon  the  subject  matter 
experts  and  the  designers  to  assist.  Continued  user  feedback  and  involvement  by  the  designers  is 
crucial  to  improve  the  individual  system  components  and  for  the  continued  success  of  the 
Tactical  EMR. 

Ongoing  training  is  also  paramount  for  success  of  the  Tactical  EMR.  Initial  training  and 

\ 

follow-up  support  is  good,  but  it  is  usually  not  enough.  Training  for  medical  personnel  must  be 
built  into  the  unit  training  schedules  on  a  continual  basis,  especially  as  the  Army’s  proposed 
EMR  becomes  fully  operational.  Medical  personnel  must  train  like  they  will  fight  and  they  must 
have  a  complete  understanding  of  the  systems  they  are  using  and  how  they  are  supposed  to  be 
used.  In  addition  to  hands  on  training,  users  must  clearly  understand  the  functionality,  purpose, 
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and  differences  between  the  Western  Regional  Tactical  EMR  and  the  Army’s  proposed 
architecture.  “Systems  that  run  from  multiple  databases  often  create  unnecessary  complexity” 
(Kirkley  &  Rewick,  2003,  649).  It  is  difficult  to  predict  when  the  Army’s  proposed  architecture 
will  be  fully  operational,  so  delineating  the  role  of  each  system  will  help  eliminate  confusion  and 
reduce  unnecessary  complexity.  It  will  also  help  reduce  over  reliance  on  the  Western  Regional 
Tactical  EMR  and  user  resistance  to  the  Army’s  architecture  when  it  is  fully  implemented. 

Training  with  the  Tactical  EMR  extends  beyond  internal  unit  and  user  training.  Training 
must  simultaneously  occur  at  and  with  the  different  echelons  of  medical  care.  Users  commented 
that  they  stopped  using  the  Tactical  EMR  because  higher  levels  of  medical  care  did  not  have  the 
same  levels  of  access  to  the  Tactical  EMR  as  they  did.  It  was  not  guaranteed  that  higher  levels 
of  medical  care  had  PIC  readers  or  access  to  ICDB  and/or  CHCS  II-T  information.  Fielding  of 
the  Tactical  EMR  must  now  extend  to  higher  levels  of  medical  care  to  promote  continued  use  at 
lower  levels  and  to  fully  assess  the  capabilities  of  the  Tactical  EMR  throughout  the  different 
echelons  of  medical  care.  Simultaneous  fielding  of  the  Tactical  EMR  with  a  combat  support 
hospital,  other  medical  centers,  or  like  units  should  improve  communication  and  collaboration 
between  the  SBCTs  and  other  elements  within  the  military  health  system.  It  will  allow  the  units 
to  expand  their  use  and  training  with  the  Tactical  EMR  and  will  allow  them  to  further  train  as 
they  will  fight  on  the  battlefield. 

Perhaps  the  greatest  factor  affecting  the  success  of  Tactical  EMR  is  command  support. 
Leadership  at  all  levels  must  fully  understand  the  capabilities  of  the  Tactical  EMR  and  its  ability 
to  improve  healthcare  for  Soldiers.  The  Tactical  EMR  must  be  marketed  to  the  combat  forces  as 
a  direct  war  fighting  system,  because  to  the  medical  personnel  it  is.  The  Tactical  EMR  provides 
medical  personnel  with  the  battlefield  tools  to  sustain  the  combat  forces.  It  also  provides  them 
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with  the  tools  to  provide  quality  healthcare  in  a  number  of  operational  environments.  Command 
emphasis  is  important  to  ensure  medical  personnel  get  the  resources  needed  and  that  they  are 
trained  with  those  resources  to  operate  at  maximum  capacity.  Without  a  “unit  champion,”  the 
concept  and  the  operation  of  the  Tactical  EMR  will  not  be  able  to  reach  its  maximum  potential. 

Conclusion 

The  development  and  use  of  an  electronic  medical  record  is  the  way  of  the  future  for  both  the 
civilian  and  military  health  systems.  Over  the  past  year  and  a  half,  MAMC,  the  SBCTs,  TMIP, 
MC4,  and  TATRC  have  made  significant  progress  in  the  development  and  implementation  of  a 
Tactical  EMR.  Even  as  this  study  was  being  completed,  continuous  daily  improvements  have 
positively  impacted  the  Tactical  EMR  and  its  ability  to  improve  quality  of  care  and  real-time 
medical  information.  This  study  provides  both  a  quantitative  and  qualitative  evaluation  of  the 
success  of  the  Western  Regional  Tactical  EMR  and  its  ability  to  improve  medical 
documentation.  It  serves  as  springboard  and  model  for  continued  evaluation  of  the  Tactical 
EMR,  especially  when  1st' Brigade  returns  from  Iraq.  The  results  indicate  that  Western  Regional 
Tactical  EMR  is  an  advantageous  interim  solution  and  model  for  the  Army  Tactical  EMR  as  it 
continues  to  be  developed.  The  results  also  establish  a  roadmap  which  provides  leadership  with 
recommended  areas  of  improvement  in  order  to  enhance  the  future  implementation  of  the 
Tactical  EMR.  The  advances  made  with  the  introduction  of  the  Tactical  EMR  are  a  step  forward 
in  providing  a  seamless  digital  healthcare  infrastructure  and  protecting  the  military’s  best  asset, 
the  Soldier,  Sailor,  Airman,  and  Marine. 
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Appendix  A.  Survey  Cover  Letter 


REPLY  TO 
ATTENTION  OF 


DEPARTMENT  OF  THE  ARMY 

WESTERN  REGIONAL  MEDICAL  COMMAND 
AND  MADIGAN  ARMY  MEDICAL  CENTER 
TACOMA,  WA  98431-1100 


MCHJ-CS 


10  January  2005 


MEMORANDUM  FOR  Medical  Personnel  Assigned  to  the  3rd  Brigade,  2nd  Infantry  Division 
SUBJECT:  Stryker  Brigade  Combat  Team  Medical  Information  System  Survey 


1 .  Attached  is  a  survey  designed  to  collect  information  for  a  study  evaluating  the  success  of  the 
Stryker  Brigade  Combat  Team  medical  information  system,  or  tactical  electronic  medical  record 
(tactical  EMR),  and  its  ability  to  improve  medical  documentation.  For  this  study,  the  concept  of  the 
tactical  EMR  includes  the  following  components:  the  personal  information  carrier  (PIC),  the 
Battlefield  Medical  Information  System-Tactical  (BMIS-T),  CHCS  II-Theater,  and  the  Integrated 
Clinical  Database  system  (ICDB).  An  honest  evaluation  of  the  tactical  EMR  is  important  and  will 
provide  valuable  insight  into  improving  its  development  and  implementation  Army  wide.  The 
ability  to  deploy  a  fully  functional  tactical  EMR  will  enhance  point-of-care  treatment,  establish  a 
longitudinal  health  record,  improve  quality  of  care,  and,  ultimately,  provide  superior  health  care  for 
our  Soldiers.  Therefore,  I  request  your  assistance  in  completing  the  attached  survey. 

2.  The  survey  has  29  questions  and  will  take  less  than  10  minutes  to  complete.  When  completing 
the  survey,  consider  the  components  of  the  tactical  EMR  and  answer  the  questions  according  to 
your  experiences  using  the  medical  information  system  when  it  was  available  and  working.  I 
encourage  you  to  add  comments  at  the  end  that  you  feel  are  important  regarding  the  system  and  any 
suggested  improvements.  Please  answer  the  questions  to  the  best  of  your  ability.  After  completing 
the  survey,  return  it  to  the  person  assigned  by  your  unit  to  collect  the  surveys.  All  surveys  are 
anonymous  and  your  input  will  be  combined  with  other  responses.  The  data  will  be  analyzed  to 
evaluate  the  success  of  the  tactical  EMR  and  to  provide  leadership  with  recommendations  for 
improving  its  development  and  implementation. 

3.  The  validity  of  the  results  depends  on  obtaining  a  high  response  rate,  so  I  thank  you  in  advance 
for  your  time  and  assistance  in  completing  the  survey.  If  you  have  questions  or  would  like  to 
submit  additional  comments,  please  contact  the  officer  responsible,  CPT  Scott  Stokoe  at  (253)  968- 
3226  or  scott.stokoe@us.amiv.mil. 


SCOTT  J.  STOKOE 
CPT,  MS 

Administrative  Resident 
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Appendix  B.  Approved  Survey 

SBCT  Medical  Information  System  Survey* 

*  The  Medical  Information  System  consists  of  the  Personal  Information  Carrier  (PIC),  BMIS-T,  CHCS  ll-T,  and  ICDB 

1  What  unit  are  you  assigned  to?  _ 

2  What  is  your  rank?  _ 

3  What  is  your  current  duty  position?  (circle)  Medic  PA  Physician  Admin  Other: 

4  How  long  have  you  been  assigned  with  your  unit?  _ months 


Please  rate  your  agreement  or  disagreement  to  the  following  statements  by  circling  the  appropriate  number  on  the  scale  provided 


[When  the  Medical  Information  System  is  working  and 
i  available,  1  find  that... 

Strongly 

Disagree 

Disagree 

Somewhat 

Disagree 

Neither 
Agree  nor 
Disagree 

Somewhat 

Agree 

Agree 

Strongly 

Agree 

5  Using  the  SBCT  Medical  Information  System  to  document  patient  care  is  easier 
than  previous  methods  of  documentation 

1 

2 

,wn. 

3 

4 

5 

6 

7 

6  Transferring  patient  encounters  between  system  components  is  easy 

83om 

,  JJPBDLIW, .. 

imamm.  . 

'  6 

7  The  SBCT  Medical  Information  System  provides  the  information  1  need,  when  1 
need  it 

i 

2 

3 

4 

5 

6 

7 

8  The  SBCT  Medical  Information  System  improves  my  ability  to  document  medical 
information 

1 

2 

3 

4 

5 

6 

7 

9  Medical  information  is  more  complete  because  of  the  SBCT  Medical  Information 
System 

i 

2 

3 

4 

5 

6 

7 

10  Accuracy  of  medical  information  is  improved  because  of  the  SBCT  Medical 
Information  System 

i 

2 

3 

4 

5 

6 

7 

1 1  The  SBCT  Medical  Information  System  provides  enough  information  to  provide 
quality  patient  care 

1 

2 

3 

4 

5 

6 

7 

12  Information  in  the  SBCT  Medical  Information  System  is  easy  to  read  and 
understand 

i 

2 

3 

4 

5 

6 

7 

13  The  SBCT  Medical  Information  System  has  enabled  me  to  create  and  store  more 
patient  encounters 

i 

2 

3 

4 

5 

6 

7 

14  1  use  the  SBCT  Medical  Information  System  every  time  1  interact  with  a  patient 

jmaamku 

uamaj 

4 

miULUJ  . 

7 

15  Documenting  medical  information  in  the  SBCT  Medical  Information  System  is 
quicker  than  documenting  information  in  paper  records 

1 

2 

3 

4 

5 

6 

7 

16  The  SBCT  Medical  Information  System  improves  medical  documentation  under 
stressful,  fast-paced  battlefield  conditions 

1 

2 

3 

4 

5 

6 

7 

17  The  SBCT  Medical  Information  System  improves  medical  documentation  in  fixed 
medical  treatment  facilities 

1 

2 

3 

4 

5 

6 

7 

18  The  SBCT  Medical  Information  System  is  an  improvement  over  paper-based 
records 

1 

2 

3 

4 

5 

6 

7 

19  The  SBCT  Medical  Information  System  is  worth  the  time  and  effort  required  to  use  it 

jmmmu. 

2 

4 

5 

6 

7 

20  Medical  information  stored  in  the  SBCT  Medical  Information  System  is  easier  to 
retrieve  than  information  stored  in  paper-based  records 

1 

2 

3 

4 

5 

6 

mmmm. 

7 

21  1  use  the  SBCT  Medical  Information  System  more  often  because  it  is  user  friendly 

smmm 

3 

4 

.JHHBU. 

7 

22  The  SBCT  Medical  Information  System  has  made  my  job/mission  easier 

'“T”" 

2 

4 

5 

6 

7 

23  The  SBCT  Medical  Information  System  has  improved  my  ability  to  provide  better 
medical  care 

1 

2 

3 

4 

5 

6 

7 

24  The  increased  availability  of  medical  information  has  improved  my  overall 
performance  level 

1 

2 

3 

4 

5 

6 

7 

25  The  SBCT  Medical  Information  System  has  improved  the  communication  of  medical 
information  from  the  point-of-care  to  higher  levels  of  care 

1 

2 

3 

4 

5 

6 

7 

26  Access  to  more  medical  information  has  improved  the  continuity  of  care  for  patients 

mmmm. 

2 

.-IHIHIi. 

4 

5 

&amuu 

27  Using  the  SBCT  Medical  Information  System  has  increased  medical  documentation 
within  my  unit 

1 

2 

3 

4 

5 

6 

7 

28  The  increase  in  medical  documentation  has  had  a  positive  impact  on  patient  care 

1 

. 2 . 

3 

4 

5 

29  1  believe  the  implementation  of  the  SBCT  Medical  Information  System  is 
progressing  in  the  right  direction 

1 

2 

3 

4 

5 

6 

7 

Comments  (If  needed,  please  continue  your  comments  on  the  back  of  this  sheet): 
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Question 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 
29 


Appendix  C.  Questions  Associated  with  Each  Variable  and  Key  Attribute 


Variable _ 

System  Quality 

Using  the  SBCT  Medical  Information  System  to  document  patient  care  is  easier  than  previous  methods 
of  documentation 

Transferring  patient  encounters  between  system  components  is  easy 

The  SBCT  Medical  Information  System  provides  the  information  I  need,  when  I  need  it 

The  SBCT  Medical  Information  System  improves  my  ability  to  document  medical  information 

Information  Quality 

Medical  information  is  more  complete  because  of  the  SBCT  Medical  Information  System 
Accuracy  of  medical  information  is  improved  because  of  the  SBCT  Medical  Information  System 
The  SBCT  Medical  Information  System  provides  enough  information  to  provide  quality  patient  care 
Information  in  the  SBCT  Medical  Information  System  is  easy  to  read  and  understand 

Usage 

The  SBCT  Medical  Information  System  has  enabled  me  to  create  and  store  more  patient  encounters 


Key  Attributes 

Ease  of  Use 
Ease  of  Use 
Response  Time 
Timesavings 

Completeness 
Accuracy  of  Data 
Accuracy  of  Data 
Legibility 

Number  of  Entries 


I  use  the  SBCT  Medical  Information  System  every  time  I  interact  with  a  patient  Frequency  of  Use 

Documenting  medical  information  in  the  SBCT  Medical  Information  System  is  quicker  than  Duration  of  Use 

documenting  information  in  paper  records 


User  Satisfaction 

The  SBCT  Medical  Information  System  improves  medical  documentation  under  stressful,  fast-paced 
battlefield  conditions 

The  SBCT  Medical  Information  System  improves  medical  documentation  in  Fixed  medical  treatment 
facilities 

The  SBCT  Medical  Information  System  is  an  improvement  over  paper-based  records 

The  SBCT  Medical  Information  System  is  worth  the  time  and  effort  required  to  use  it 

Medical  information  stored  in  the  SBCT  Medical  Information  System  is  easier  to  retrieve  than 
information  stored  in  paper-based  records 

I  use  the  SBCT  Medical  Information  System  more  because  it  is  user  friendly 

Individual  Impact 

The  SBCT  Medical  Information  System  has  made  my  job/mission  easier 

The  SBCT  Medical  Information  System  has  improved  my  ability  to  provide  better  medical  care 
The  increased  availability  of  medical  information  has  improved  my  overall  performance  level 

Organizational  Impact 

The  SBCT  Medical  Information  System  has  improved  the  communication  of  medical  information  from 
the  point-of-care  to  higher  levels  of  care 

Access  to  more  medical  information  has  improved  the  continuity  of  care  for  patients 

Using  the  SBCT  Medical  Information  System  has  increased  medical  documentation  within  my  unit 

The  increase  in  medical  documentation  has  had  a  positive  impact  on  patient  care 

I  believe  the  implementation  of  the  SBCT  Medical  Information  System  is  progressing  in  the  right 
direction 


User  Satisfaction 

User  Satisfaction 

User  Satisfaction 
Attitude 

User  Friendliness 

User  Friendliness 

Change  in  Work  Patterns 
Change  in  Work  Patterns 
Direct  Benefits 

Communication  / 
Collaboration 

Impact  on  Patient  Care 

Impact  on  Patient  Care 

Impact  on  Patient  Care 
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Appendix  D.  Descriptive  Statistics  by  Unit 


All  Surveys 

3rd  Brigade 

1  st  Brigade 

Standard 

Standard 

Standard 

Variable 

n 

Mean 

Deviation 

n 

Mean 

Deviation 

n 

Mean  Deviation 

Months  Assigned 

134 

25.75 

13.97 

90 

28.32 

14.81 

44 

20.48 

10.34 

Question 

5 

134 

2.95 

1.59 

90 

3.06 

1.60 

44 

2.73 

1.55 

6 

135 

3.06 

1.52 

90 

3.11 

1.59 

45 

2.96 

1.38 

7 

134 

3.33 

1.51 

90 

3.38 

1.53 

44 

3.23 

1.48 

8 

135 

3.57 

1.67 

90 

3.53 

1.68 

45 

3.64 

1.68 

9 

135 

3.43 

1.64 

90 

3.23 

1.56 

45 

3.82 

1.75 

10 

134 

3.51 

1.55 

90 

3.37 

1.54 

44 

3.80 

1.55 

11 

135 

3.71 

1.48 

90 

3.70 

1.53 

45 

3.73 

1.39 

12 

135 

3.75 

1.60 

90 

3.67 

1.61 

45 

3.91 

1.59 

13 

134 

3.45 

1.62 

89 

3.20 

1.51 

45 

3.93 

1.72 

14 

135 

2.80 

1.71 

90 

2.58 

1.61 

45 

3.24 

1.82 

15 

135 

2.68 

1.65 

90 

2.93 

1.70 

45 

2.18 

1.44 

16 

135 

2.43 

1.56 

90 

2.54 

1.55 

45 

2.20 

1.58 

17 

134 

4.08 

1.69 

90 

3.87 

1.66 

44 

4.52 

1.66 

18 

134 

3.49 

1.56 

89 

3.48 

1.58 

45 

3.49 

1.55 

19 

135 

3.27 

1.62 

90 

3.30 

1.60 

45 

3.20 

1.66 

20 

135 

3.67 

1.69 

90 

3.40 

1.62 

45 

4.22 

1.72 

21 

133 

2.83 

1.45 

88 

2.86 

1.45 

45 

2.76 

1.46 

22 

135 

2.95 

1.50 

90 

3.00 

1.53 

45 

2.84 

1.46 

23 

134 

3.04 

1.54 

89 

3.07 

1.54 

45 

2.98 

1.57 

24 

135 

3.10 

1.52 

90 

3.09 

1.54 

45 

3.11 

1.48 

25 

134 

3.28 

1.65 

90 

3.17 

1.62 

44 

3.52 

1.72 

26 

133 

3.55 

1.62 

88 

3.36 

1.62 

45 

3.91 

1.56 

27 

134 

3.24 

1.52 

89 

3.11 

1.46 

45 

3.49 

1.62 

28 

135 

3.38 

1.59 

90 

3.20 

1.52 

45 

3.73 

1.68 

29 

135 

3.79 

1.72 

90 

3.97 

1.78 

45 

3.44 

1.56 

Key  Attribute 

Ease  of  Use 

135 

3.00 

1.45 

90 

3.08 

1.51 

45 

2.84 

1.33 

Response  Time 

134 

3.33 

1.51 

90 

3.38 

1.53 

44 

3.23 

1.48 

Timesavings 

135 

3.57 

1.67 

90 

3.53 

1.68 

45 

3.64 

1.68 

Completeness 

135 

3.43 

1.64 

90 

3.23 

1.56 

45 

3.82 

1.75 

Accuracy  of  Data 

135 

3.61 

1.40 

90 

3.53 

1.43 

45 

3.78 

1.34 

Legibility 

135 

3.75 

1.60 

90 

3.67 

1.61 

45 

3.91 

1.59 

Number  of  Entries 

134 

3.45 

1.62 

89 

3.20 

1.51 

45 

3.93 

1.72 

Frequency  of  Use 

135 

2.80 

1.71 

90 

2.58 

1.61 

45 

3.24 

1.82 

Duration  of  Use 

135 

2.68 

1.65 

90 

2.93 

1.70 

45 

2.18 

1.44 

user  satisfaction 

135 

3.33 

1.33 

90 

3.30 

1.38 

45 

3.39 

1.22 

Attitude 

135 

3.27 

1.62 

90 

3.30 

1.60 

45 

3.20 

1.66 

User  Friendliness 

135 

3.27 

1.41 

90 

3.16 

1.46 

45 

3.49 

1.30 

Change  in  Work  Patterns 

135 

2.99 

1.43 

90 

3.03 

1.45 

45 

2.91 

1.39 

Direct  Benefits 

135 

3.10 

1.52 

90 

3.09 

1.54 

45 

3.11 

1.48 

Communication/Collaboration 

134 

3.28 

1.65 

90 

3.17 

1.62 

44 

3.52 

1.72 

Impact  on  Patient  Care 

135 

3.39 

1.45 

90 

3.24 

1.41 

45 

3.71 

1.49 

Variable 

System  Quality 

135 

3.31 

1.38 

90 

3.33 

1.43 

45 

3.26 

1.31 

Information  Quality 

135 

3.60 

1.40 

90 

3.48 

1.43 

45 

3.84 

1.33 

Usage 

135 

2.98 

1.37 

90 

2.91 

1.41 

45 

3.12 

1.29 

User  Satisfaction 

135 

3.29 

1.35 

90 

3.25 

1.39 

45 

3.36 

1.25 

Individual  Impact 

135 

3.04 

1.40 

90 

3.06 

1.45 

45 

3.01 

1.33 

Organizational  Impact 

135 

3.35 

1.47 

90 

3.20 

1.44 

45 

3.64 

1.49 
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Appendix  E.  Ranking  of  Variables  and  Key  Attributes  by  Demographic  Category 


Overall 

'  Unit 

Rank 

Duty  Position 

Time  with  Unit 

All 

3rd 

1st 

Physician  / 

0-  12 

12-24 

24  + 

Variable  and  Key  Attribute 

Surveys 

Brigade  Brigade 

Enlisted  NCO  Officer 

Admin 

Medic 

Months  Months  Months 

System  Quality 

3 

2 

4 

2 

5 

3 

3 

3 

5 

2 

2 

Ease  of  Use 

13 

13 

15 

12 

14 

14 

15 

13 

15 

12 

13 

Response  Time 

7 

4 

11 

2 

12 

9 

8 

6 

12 

5 

6 

Timesavings 

3 

2 

6 

3 

3 

4 

5 

3 

5 

3 

3 

Information  Quality 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

Completeness 

5 

8 

3 

5 

8 

1 

1 

7 

2 

4 

10 

Accuracy  of  Data 

2 

3 

4 

4 

2 

2 

2 

2 

6 

2 

2 

Legibility 

1 

1 

2 

1 

1 

3 

3 

1 

3 

1 

1 

Usage 

6 

6 

5 

5 

6 

5 

5 

6 

3 

6 

6 

Number  of  Entries 

4 

9 

1 

6 

4 

5 

4 

4 

1 

7 

4 

Frequency  of  Use 

15 

16 

10 

15 

16 

10 

11 

16 

7 

16 

16 

Duration  of  Use 

16 

15 

16 

16 

15 

16 

16 

15 

16 

15 

15 

User  Satisfaction 

4 

3 

3 

4 

3 

4 

4 

4 

4 

3 

3 

User  Satisfaction 

8 

5 

9 

10 

6 

7 

7 

9 

10 

8 

5 

Attitude 

10 

6 

12 

11 

9 

13 

13 

10 

13 

10 

8 

User  Friendliness 

11 

11 

8 

7 

11 

8 

10 

11 

9 

9 

11 

Individual  Impact 

5 

5 

6 

6 

4 

6 

6 

5 

6 

5 

5 

Change  in  Work  Patterns 

14 

14 

14 

14 

13 

15 

14 

14 

14 

14 

14 

Direct  Benefits 

12 

12 

13 

13 

10 

11 

9 

12 

11 

13 

12 

Organizational  Impact 

2 

4 

2 

3 

2 

2 

2 

2 

2 

4 

4 

Communication/Collaboration 

9 

10 

7 

8 

7 

12 

12 

8 

4 

11 

8 

Impact  on  Patient  Care 

6 

7 

5 

9 

5 

6 

6 

5 

8 

6 

7 
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Appendix  F.  Spearman  Rho  Correlation  Analysis  for  Variables 


System 

Quality 

Information 

Quality 

Usage 

User 

Satisfaction 

Individual  Organizational 
Impact  Impact 

Duty 

Position 

Unit 

Time 

with  Unit 

Rank 

System  Quality 

1 

.80(**) 

,78(**) 

,84(**) 

.77(**) 

.690) 

.04 

-.03 

-.17 

-.14 

Information  Quality 

,80(**) 

1 

.74(**) 

,76(**) 

.73(**) 

.72(**)  ' 

.10 

-.01 

-,24(**) 

.04 

Usage 

.78(**) 

,74(**) 

1 

.79(**) 

.690) 

.03 

.02 

-,31(**) 

-.10 

User  Satisfaction 

,84(**) 

,76(**) 

.79(**) 

1 

.86(**) 

,73(**) 

.10 

.03 

-,18(*) 

-.10 

Individual  Impact 

,77(**) 

.73(**) 

.76(**) 

,86(**) 

1 

.79(**) 

.07 

.01 

-,18(*) 

-.06 

Organizational  Impact 

.69(**) 

,72(**) 

.69(**) 

.73(**) 

.79(**) 

1 

.04 

.02 

-.25(**) 

-.04 

Duty  Position 

.04 

.10 

.03 

.10 

.07 

.04 

1.00 

.03 

-.08 

.17 

Unit 

-.03 

-.01 

.02 

.03 

.01 

.02 

.03 

1.00 

-.05 

-.09 

Time  wth  Unit 

-.17 

-.24(**) 

-.31(**) 

-,18(*) 

-.18(*) 

-,25(**) 

-.08 

-.05 

1 

.05 

Rank 

-.14 

.04 

-.10 

-.10 

-.06 

-.04 

.17 

-.09 

.05 

1 

Note:  N=  132;  **  p  <  .01  (2-tailed);  p  <  .05  (2-tailed) 
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Appendix  G.  Independent- Samples  t-test  for  Unit  and  Duty  Position 


Variable 

Unit 

Duty  Position 

t 

Sig. 

t 

Sig. 

System  Quality 

.30 

.77 

-.34 

.74 

Information  Quality 

-1.41 

.16 

1.01 

.32 

Usage 

-.84 

.40 

.14 

.89 

User  Satisfaction 

-.44 

.66 

-.40 

.69 

Individual  Impact 

.18 

.86 

-.41 

.68 

Organizational  Impact 

-1.67 

.10 

-.11 

.92 

Note:  N  =  133;  alpha  =  .05 
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Appendix  H.  Analysis  of  Variance  for  Rank  Groups 


Variable 

Sum  of  Squares  Mean  Square 

F 

Sig. 

System  Quality 

Between  Groups 

1.62 

.81 

.42 

.66 

Within  Groups 

248.51 

1.93 

Total 

250.13 

Information  Quality 

Between  Groups 

2.33 

1.16 

.60 

.56 

Within  Groups 

254.31 

1.97 

Total 

256.64 

Usage 

Between  Groups 

.32 

.16 

.10 

.92 

Within  Groups 

242.67 

1.88 

Total 

242.67 

User  Satisfaction 

Between  Groups 

.22 

.11 

.10 

.94 

Within  Groups 

237.97 

1.85 

Total 

237.97 

Individual  Impact 

Between  Groups 

1.65 

0.83 

.41 

.66 

Within  Groups 

259.74 

2.01 

Total 

261.39 

Organizational  Impact 

Between  Groups 

.53 

.27 

.12 

.89 

Within  Groups 

279.98 

2.17 

Total 

279.98 

Note:  df=  2,129;  alpha  =  .05 
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